2023-10-07T22:48:23.260 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-07T22:48:23.298 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} --name yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/orig.config.yaml 2023-10-07T22:48:23.352 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T22:48:23.434 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} email: yweinste@redhat.com first_in_suite: false job_id: '7418034' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 kclient: mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-07-1508-quincy sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 owner: scheduled_yuriw@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-07-1508-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3iQMWZ//AVUYsGaoIawOqjB+7cK8RVEAe5L5kvT9lAPyUCOkuh8SHyaWo+9X83DY15w2jV4sIrN2MgSyzCL5E= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFRLp2JeOCbBLAfSQ/QSN91MIiIotZlxeJx7RvTI6mYzKBrV3Ngh9zhXujL2kieOSDrPoYTZyBx9o3gWn2IJM4M= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxdMWiYGXZ9jBVTUMzUA/sdy5F4FxcZoC2B/rds4X0S5vRwRjolYZD6OUy6XBHJfRxzaJwl0rSqzpl80/k/r7Q= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - check-counter: {} - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_22:15:20 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-07T22:48:23.435 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa; will attempt to use it 2023-10-07T22:48:23.436 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks 2023-10-07T22:48:23.436 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-07T22:48:23.438 INFO:teuthology.task.internal:Checking packages... 2023-10-07T22:48:23.461 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-07T22:48:23.461 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-07T22:48:23.461 INFO:teuthology.packaging:ref: None 2023-10-07T22:48:23.461 INFO:teuthology.packaging:tag: None 2023-10-07T22:48:23.461 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T22:48:23.462 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T22:48:23.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T22:48:23.611 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320.g08cac018 2023-10-07T22:48:23.612 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-07T22:48:23.619 INFO:teuthology.task.internal:no buildpackages task found 2023-10-07T22:48:23.619 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-07T22:48:23.633 INFO:teuthology.task.internal:Saving configuration 2023-10-07T22:48:23.651 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-07T22:48:23.657 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T22:48:23.681 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 22:43:02.410449', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3iQMWZ//AVUYsGaoIawOqjB+7cK8RVEAe5L5kvT9lAPyUCOkuh8SHyaWo+9X83DY15w2jV4sIrN2MgSyzCL5E='} 2023-10-07T22:48:23.707 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 22:43:02.411961', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFRLp2JeOCbBLAfSQ/QSN91MIiIotZlxeJx7RvTI6mYzKBrV3Ngh9zhXujL2kieOSDrPoYTZyBx9o3gWn2IJM4M='} 2023-10-07T22:48:23.726 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 22:43:02.408701', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxdMWiYGXZ9jBVTUMzUA/sdy5F4FxcZoC2B/rds4X0S5vRwRjolYZD6OUy6XBHJfRxzaJwl0rSqzpl80/k/r7Q='} 2023-10-07T22:48:23.726 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-07T22:48:23.732 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-07T22:48:23.732 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-07T22:48:23.732 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-07T22:48:23.732 INFO:teuthology.run_tasks:Running task console_log... 2023-10-07T22:48:23.826 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdc80c7ab80>, signals=[15]) 2023-10-07T22:48:23.826 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-07T22:48:23.832 INFO:teuthology.task.internal:Opening connections... 2023-10-07T22:48:23.832 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T22:48:23.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:48:23.904 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2023-10-07T22:48:23.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:48:23.972 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-10-07T22:48:23.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:48:24.039 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-07T22:48:24.047 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2023-10-07T22:48:24.065 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2023-10-07T22:48:24.066 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2023-10-07T22:48:24.123 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Red Hat Enterprise Linux" 2023-10-07T22:48:24.123 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8.4 (Ootpa)" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:ID="rhel" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="fedora" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8.4" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.redhat.com/" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T22:48:24.124 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T22:48:24.125 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-07T22:48:24.125 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-07T22:48:24.125 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-07T22:48:24.125 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2023-10-07T22:48:24.156 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2023-10-07T22:48:24.173 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2023-10-07T22:48:24.173 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.4 (Ootpa)" 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.4" 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2023-10-07T22:48:24.229 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-07T22:48:24.230 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-07T22:48:24.231 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2023-10-07T22:48:24.247 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-10-07T22:48:24.265 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-10-07T22:48:24.265 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-10-07T22:48:24.321 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-10-07T22:48:24.322 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.4 (Ootpa)" 2023-10-07T22:48:24.322 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-10-07T22:48:24.322 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-10-07T22:48:24.322 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.4" 2023-10-07T22:48:24.322 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-10-07T22:48:24.322 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-07T22:48:24.323 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-07T22:48:24.324 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-10-07T22:48:24.347 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-07T22:48:24.358 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-07T22:48:24.364 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-07T22:48:24.364 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2023-10-07T22:48:24.367 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2023-10-07T22:48:24.368 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-10-07T22:48:24.383 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-07T22:48:24.391 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-07T22:48:24.391 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2023-10-07T22:48:24.423 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2023-10-07T22:48:24.425 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-10-07T22:48:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-07T22:48:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-07T22:48:24.441 INFO:teuthology.orchestra.run.smithi123.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-07T22:48:24.442 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-07T22:48:24.513 INFO:teuthology.run_tasks:Running task kernel... 2023-10-07T22:48:24.527 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-07T22:48:24.527 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-07T22:48:24.527 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-07T22:48:24.528 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T22:48:24.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T22:48:24.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-07T22:48:24.786 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-07T22:48:24.787 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-07T22:48:24.787 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-10-07T22:48:24.804 INFO:teuthology.orchestra.run.smithi067.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-07T22:48:24.805 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-07T22:48:24.805 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-07T22:48:24.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum upgrade -y linux-firmware 2023-10-07T22:48:26.350 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-07T22:48:26.351 INFO:teuthology.orchestra.run.smithi067.stdout:Unable to read consumer identity 2023-10-07T22:48:27.747 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-07T22:48:28.263 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 33 MB/s | 16 MB 00:00 2023-10-07T22:48:31.761 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-07T22:48:32.664 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-10-07T22:48:32.666 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-10-07T22:48:32.666 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-10-07T22:48:32.806 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-07T22:48:32.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-07T22:48:32.951 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-07T22:48:32.951 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-07T22:48:32.953 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi067.front.sepia.ceph.com to complete... 2023-10-07T22:48:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07 22:48:47 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-07T22:48:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:FINISHED --2023-10-07 22:48:47-- 2023-10-07T22:48:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:Total wall clock time: 15s 2023-10-07T22:48:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:Downloaded: 1 files, 363M in 14s (25.4 MB/s) 2023-10-07T22:48:47.591 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-07T22:48:47.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-07T22:48:49.401 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying... ######################################## 2023-10-07T22:48:49.796 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing... ######################################## 2023-10-07T22:48:49.918 INFO:teuthology.orchestra.run.smithi067.stdout:Updating / installing... 2023-10-07T22:49:30.449 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-07T22:49:51.992 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-07T22:49:51.993 DEBUG:teuthology.orchestra.run.smithi067:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-07T22:49:52.044 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.044 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.044 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-07T22:49:52.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-07T22:49:52.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-07T22:49:52.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-07T22:49:52.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-07T22:49:52.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-07T22:49:52.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-07T22:49:52.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-07T22:49:52.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-07T22:49:52.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-07T22:49:52.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-07T22:49:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-07T22:49:52.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-07T22:49:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-07T22:49:52.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-07T22:49:52.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-07T22:49:52.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-07T22:49:52.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-07T22:49:52.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-07T22:49:52.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-07T22:49:52.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-07T22:49:52.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-07T22:49:52.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-07T22:49:52.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-07T22:49:52.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-07T22:49:52.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-07T22:49:52.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-07T22:49:52.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-07T22:49:52.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-07T22:49:52.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-07T22:49:52.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-07T22:49:52.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-07T22:49:52.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-07T22:49:52.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-07T22:49:52.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-07T22:49:52.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-07T22:49:52.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-07T22:49:52.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-07T22:49:52.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-07T22:49:52.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-07T22:49:52.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-07T22:49:52.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-07T22:49:52.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-07T22:49:52.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-07T22:49:52.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-07T22:49:52.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-07T22:49:52.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-07T22:49:52.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-07T22:49:52.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-07T22:49:52.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-07T22:49:52.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-07T22:49:52.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-07T22:49:52.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-07T22:49:52.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-07T22:49:52.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-07T22:49:52.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-07T22:49:52.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-07T22:49:52.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-07T22:49:52.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-07T22:49:52.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-07T22:49:52.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-07T22:49:52.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-07T22:49:52.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-07T22:49:52.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-07T22:49:52.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-07T22:49:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-07T22:49:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-07T22:49:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-07T22:49:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-07T22:49:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-07T22:49:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-07T22:49:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-07T22:49:52.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-07T22:49:52.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-07T22:49:52.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-07T22:49:52.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-07T22:49:52.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-07T22:49:52.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-07T22:49:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-07T22:49:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-07T22:49:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-07T22:49:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-07T22:49:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-07T22:49:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-07T22:49:52.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-07T22:49:52.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-07T22:49:52.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-07T22:49:52.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-07T22:49:52.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-07T22:49:52.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-07T22:49:52.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-07T22:49:52.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-07T22:49:52.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-07T22:49:52.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-07T22:49:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-07T22:49:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-07T22:49:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-07T22:49:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-07T22:49:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-07T22:49:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-07T22:49:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-07T22:49:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-07T22:49:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-07T22:49:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-07T22:49:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-07T22:49:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-07T22:49:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-07T22:49:52.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-07T22:49:52.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-07T22:49:52.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-07T22:49:52.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-07T22:49:52.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-07T22:49:52.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-07T22:49:52.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-07T22:49:52.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-07T22:49:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-07T22:49:52.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-07T22:49:52.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-07T22:49:52.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-07T22:49:52.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-07T22:49:52.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-07T22:49:52.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-07T22:49:52.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-07T22:49:52.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-07T22:49:52.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-07T22:49:52.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-07T22:49:52.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-07T22:49:52.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-07T22:49:52.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-07T22:49:52.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-07T22:49:52.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-07T22:49:52.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-07T22:49:52.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-07T22:49:52.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-07T22:49:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-07T22:49:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-07T22:49:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-07T22:49:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-07T22:49:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-07T22:49:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-07T22:49:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-07T22:49:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-07T22:49:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-07T22:49:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-07T22:49:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-07T22:49:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-07T22:49:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-07T22:49:52.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-07T22:49:52.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-07T22:49:52.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-07T22:49:52.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-07T22:49:52.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-07T22:49:52.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-07T22:49:52.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-07T22:49:52.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-07T22:49:52.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-07T22:49:52.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-07T22:49:52.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-07T22:49:52.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-07T22:49:52.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-07T22:49:52.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-07T22:49:52.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-07T22:49:52.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-07T22:49:52.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-07T22:49:52.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-07T22:49:52.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-07T22:49:52.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-07T22:49:52.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-07T22:49:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-07T22:49:52.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-07T22:49:52.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-07T22:49:52.115 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-07T22:49:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-07T22:49:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-07T22:49:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-07T22:49:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-07T22:49:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-07T22:49:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-07T22:49:52.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-07T22:49:52.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-07T22:49:52.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-07T22:49:52.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-07T22:49:52.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-07T22:49:52.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-07T22:49:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-07T22:49:52.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-07T22:49:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-07T22:49:52.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-07T22:49:52.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-07T22:49:52.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-07T22:49:52.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-07T22:49:52.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-07T22:49:52.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-07T22:49:52.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-07T22:49:52.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-07T22:49:52.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-07T22:49:52.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-07T22:49:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-07T22:49:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-07T22:49:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-07T22:49:52.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-07T22:49:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-07T22:49:52.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-07T22:49:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-07T22:49:52.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-07T22:49:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-07T22:49:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-07T22:49:52.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-07T22:49:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-07T22:49:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-07T22:49:52.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-07T22:49:52.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-07T22:49:52.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-07T22:49:52.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-07T22:49:52.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-07T22:49:52.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-07T22:49:52.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-07T22:49:52.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-07T22:49:52.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-07T22:49:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-07T22:49:52.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-07T22:49:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-07T22:49:52.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-07T22:49:52.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-07T22:49:52.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-07T22:49:52.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-07T22:49:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-07T22:49:52.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-07T22:49:52.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-07T22:49:52.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-07T22:49:52.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-07T22:49:52.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-07T22:49:52.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-07T22:49:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-07T22:49:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-07T22:49:52.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-07T22:49:52.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-07T22:49:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-07T22:49:52.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-07T22:49:52.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-07T22:49:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-07T22:49:52.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-07T22:49:52.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-07T22:49:52.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-07T22:49:52.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-07T22:49:52.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-07T22:49:52.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-07T22:49:52.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-07T22:49:52.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-07T22:49:52.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-07T22:49:52.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-07T22:49:52.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-07T22:49:52.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-07T22:49:52.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-07T22:49:52.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-07T22:49:52.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-07T22:49:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-07T22:49:52.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-07T22:49:52.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-07T22:49:52.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-07T22:49:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-07T22:49:52.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-07T22:49:52.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-07T22:49:52.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-07T22:49:52.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-07T22:49:52.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-07T22:49:52.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-07T22:49:52.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-07T22:49:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-07T22:49:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-07T22:49:52.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-07T22:49:52.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-07T22:49:52.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-07T22:49:52.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-07T22:49:52.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-07T22:49:52.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-07T22:49:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-07T22:49:52.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-07T22:49:52.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-07T22:49:52.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-07T22:49:52.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-07T22:49:52.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-07T22:49:52.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-07T22:49:52.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-07T22:49:52.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-07T22:49:52.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-07T22:49:52.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-07T22:49:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-07T22:49:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-07T22:49:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-07T22:49:52.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-07T22:49:52.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-07T22:49:52.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-07T22:49:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-07T22:49:52.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-07T22:49:52.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-07T22:49:52.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-07T22:49:52.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-07T22:49:52.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-07T22:49:52.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-07T22:49:52.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-07T22:49:52.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-07T22:49:52.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-07T22:49:52.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-07T22:49:52.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-07T22:49:52.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-07T22:49:52.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-07T22:49:52.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-07T22:49:52.238 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.238 DEBUG:teuthology.orchestra.run.smithi067:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-07T22:49:52.259 INFO:teuthology.orchestra.run.smithi067.stdout:postinstall scriptlet (using /bin/sh): 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-07T22:49:52.260 INFO:teuthology.orchestra.run.smithi067.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-07T22:49:52.261 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.261 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2023-10-07T22:49:52.261 INFO:teuthology.orchestra.run.smithi067.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-07T22:49:52.261 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-07T22:49:52.261 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.261 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2023-10-07T22:49:52.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.02 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 99.el8_4.1 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-07T22:49:52.334 INFO:teuthology.orchestra.run.smithi067.stdout:Group : System Environment/Base 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 9264631 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Relocations : (not relocatable) 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : Red Hat, Inc. 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : Red Hat, Inc. 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2023-10-07T22:49:52.335 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T22:49:52.336 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-07T22:49:52.336 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-07T22:49:52.336 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-07T22:49:52.336 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2023-10-07T22:49:52.336 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T22:49:52.336 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2023-10-07T22:49:52.337 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-07T22:49:52.337 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-07T22:49:52.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-07T22:49:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2023-10-07T22:49:59.791 INFO:teuthology.orchestra.run.smithi067.stderr:done 2023-10-07T22:49:59.794 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-10-07T22:49:59.808 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.lOt4K6vaul 2023-10-07T22:49:59.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lOt4K6vaul 2023-10-07T22:49:59.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.lOt4K6vaul 2023-10-07T22:50:00.034 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.lOt4K6vaul is 6KB 2023-10-07T22:50:00.082 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.lOt4K6vaul 2023-10-07T22:50:00.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2023-10-07T22:50:00.171 INFO:teuthology.orchestra.run.smithi067.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-07T22:50:00.171 INFO:teuthology.orchestra.run.smithi067.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-07T22:50:00.171 INFO:teuthology.orchestra.run.smithi067.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-07T22:50:00.171 INFO:teuthology.orchestra.run.smithi067.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-07T22:50:00.172 INFO:teuthology.orchestra.run.smithi067.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-07T22:50:00.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-07T22:50:00.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2023-10-07T22:50:30.340 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-07T22:50:30.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T22:50:30.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:51:04.954 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-10-07T22:51:13.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T22:51:13.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:51:32.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-10-07T22:51:44.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T22:51:44.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:51:47.418 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-10-07T22:52:02.420 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T22:52:02.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T22:52:02.845 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-10-07T22:52:03.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T22:52:03.241 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-07T22:52:03.241 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-10-07T22:52:03.277 INFO:teuthology.orchestra.run.smithi067.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-07T22:52:03.277 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-07T22:52:03.277 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T22:52:03.277 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-07T22:52:04.279 DEBUG:teuthology.parallel:result is None 2023-10-07T22:52:04.279 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-07T22:52:04.286 INFO:teuthology.task.internal:Creating test directory... 2023-10-07T22:52:04.286 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T22:52:04.291 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T22:52:04.293 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T22:52:04.326 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-07T22:52:04.332 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-07T22:52:04.379 INFO:teuthology.task.internal:Creating archive directory... 2023-10-07T22:52:04.380 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T22:52:04.384 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T22:52:04.386 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T22:52:04.432 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-07T22:52:04.439 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-07T22:52:04.439 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T22:52:04.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T22:52:04.478 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T22:52:04.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T22:52:04.493 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T22:52:04.498 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T22:52:04.520 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T22:52:04.570 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T22:52:04.581 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T22:52:04.587 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T22:52:04.597 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T22:52:04.636 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T22:52:04.674 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T22:52:04.677 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-07T22:52:04.684 INFO:teuthology.task.internal:Configuring sudo... 2023-10-07T22:52:04.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T22:52:04.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T22:52:04.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T22:52:04.749 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-07T22:52:04.758 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-07T22:52:04.759 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T22:52:04.763 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T22:52:04.765 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T22:52:04.795 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T22:52:04.899 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T22:52:04.988 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T22:52:05.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T22:52:05.130 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T22:52:05.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T22:52:05.196 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T22:52:05.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T22:52:05.285 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T22:52:05.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T22:52:05.411 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T22:52:05.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T22:52:05.476 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T22:52:05.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T22:52:05.564 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T22:52:05.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T22:52:05.687 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T22:52:05.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T22:52:05.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2023-10-07T22:52:05.756 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2023-10-07T22:52:05.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-10-07T22:52:05.811 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T22:52:05.854 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T22:52:05.864 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T22:52:06.338 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-07T22:52:06.347 INFO:teuthology.task.internal:Starting timer... 2023-10-07T22:52:06.347 INFO:teuthology.run_tasks:Running task pcp... 2023-10-07T22:52:06.357 INFO:teuthology.run_tasks:Running task selinux... 2023-10-07T22:52:06.401 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-07T22:52:06.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2023-10-07T22:52:06.515 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: [ OK ] 2023-10-07T22:52:06.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2023-10-07T22:52:06.608 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: [ OK ] 2023-10-07T22:52:06.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2023-10-07T22:52:06.664 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: [ OK ] 2023-10-07T22:52:06.666 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-07T22:52:06.666 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2023-10-07T22:52:06.709 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2023-10-07T22:52:06.710 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2023-10-07T22:52:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2023-10-07T22:52:06.734 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2023-10-07T22:52:06.769 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2023-10-07T22:52:06.769 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2023-10-07T22:52:06.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T22:52:06.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T22:52:06.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T22:52:06.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T22:52:06.859 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T22:52:06.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T22:52:06.883 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-07T22:52:06.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2023-10-07T22:52:06.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2023-10-07T22:52:06.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2023-10-07T22:52:06.998 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-07T22:52:07.008 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-07T22:52:07.008 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-07T22:52:07.023 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-07T22:52:07.024 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi086.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-07T23:03:38.242 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2023-10-07T23:03:38.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T23:03:38.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:03:38.328 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-10-07T23:03:38.433 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-07T23:03:38.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-07T23:03:38.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:03:38.503 DEBUG:teuthology.orchestra.run.smithi086:> true 2023-10-07T23:03:38.571 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-07T23:03:38.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-07T23:03:38.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:03:38.640 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-10-07T23:03:38.661 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-07T23:03:38.661 INFO:teuthology.run_tasks:Running task clock... 2023-10-07T23:03:38.671 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-07T23:03:38.671 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T23:03:38.672 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:03:38.676 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T23:03:38.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:03:38.679 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T23:03:38.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:03:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-07T23:03:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-07T23:03:38.730 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-07T23:03:38.739 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-07T23:03:38.744 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-07T23:03:38.760 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2023-10-07T23:03:38.770 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2023-10-07T23:03:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2023-10-07T23:03:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-07T23:03:38.784 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-07T23:03:38.785 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2023-10-07T23:03:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-07T23:03:38.799 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-07T23:03:38.812 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-07T23:03:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2023-10-07T23:03:38.865 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2023-10-07T23:03:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:210 Number of sources = 4 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.870 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.886 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-10-07T23:03:38.888 INFO:teuthology.orchestra.run.smithi123.stdout:210 Number of sources = 4 2023-10-07T23:03:38.889 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T23:03:38.889 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-10-07T23:03:38.889 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.889 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.889 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.889 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:38.898 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-07T23:03:38.933 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-07T23:03:39.017 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2023-10-07T23:03:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:210 Number of sources = 4 2023-10-07T23:03:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T23:03:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-10-07T23:03:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:39.022 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:39.023 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:03:39.023 INFO:teuthology.run_tasks:Running task pexec... 2023-10-07T23:03:39.034 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-07T23:03:39.034 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T23:03:39.034 DEBUG:teuthology.orchestra.run.smithi067:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-07T23:03:39.034 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi086.front.sepia.ceph.com 2023-10-07T23:03:39.035 DEBUG:teuthology.orchestra.run.smithi086:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-07T23:03:39.035 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi123.front.sepia.ceph.com 2023-10-07T23:03:39.035 DEBUG:teuthology.orchestra.run.smithi123:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-07T23:03:39.422 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-10-07T23:03:39.422 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2023-10-07T23:03:39.571 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-07T23:03:40.489 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:04:46 ago on Sat 07 Oct 2023 10:58:54 PM UTC. 2023-10-07T23:03:40.500 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:48 ago on Sat 07 Oct 2023 10:58:52 PM UTC. 2023-10-07T23:03:40.756 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:53 ago on Sat 07 Oct 2023 11:01:47 PM UTC. 2023-10-07T23:03:42.503 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2023-10-07T23:03:42.504 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2023-10-07T23:03:42.504 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2023-10-07T23:03:42.504 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2023-10-07T23:03:42.504 INFO:teuthology.orchestra.run.smithi086.stdout:Resetting modules: 2023-10-07T23:03:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools 2023-10-07T23:03:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:03:42.505 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2023-10-07T23:03:42.505 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2023-10-07T23:03:42.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:03:42.510 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout:Resetting modules: 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-10-07T23:03:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:03:42.999 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-10-07T23:03:43.000 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout:Resetting modules: 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout: container-tools 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-10-07T23:03:43.001 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:03:43.534 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:51 ago on Sat 07 Oct 2023 10:58:52 PM UTC. 2023-10-07T23:03:43.560 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:04:49 ago on Sat 07 Oct 2023 10:58:54 PM UTC. 2023-10-07T23:03:44.257 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:56 ago on Sat 07 Oct 2023 11:01:47 PM UTC. 2023-10-07T23:03:47.635 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-10-07T23:03:48.062 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-10-07T23:03:48.075 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-10-07T23:03:48.236 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2023-10-07T23:03:48.675 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-07T23:03:48.696 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2023-10-07T23:03:49.403 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:56 ago on Sat 07 Oct 2023 10:58:52 PM UTC. 2023-10-07T23:03:50.372 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:04:55 ago on Sat 07 Oct 2023 10:58:54 PM UTC. 2023-10-07T23:03:50.461 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:02 ago on Sat 07 Oct 2023 11:01:47 PM UTC. 2023-10-07T23:03:51.405 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-10-07T23:03:51.453 INFO:teuthology.orchestra.run.smithi123.stdout:========================================================================================================= 2023-10-07T23:03:51.453 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-10-07T23:03:51.453 INFO:teuthology.orchestra.run.smithi123.stdout:========================================================================================================= 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout:Installing group/module packages: 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-07T23:03:51.454 INFO:teuthology.orchestra.run.smithi123.stdout:Installing module profiles: 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools/common 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools rhel8 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout:========================================================================================================= 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout:Install 7 Packages 2023-10-07T23:03:51.455 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:03:51.456 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 16 M 2023-10-07T23:03:51.456 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 57 M 2023-10-07T23:03:51.456 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-10-07T23:03:51.641 INFO:teuthology.orchestra.run.smithi123.stdout:(1/7): yajl-2.1.0-10.el8.x86_64.rpm 220 kB/s | 41 kB 00:00 2023-10-07T23:03:51.647 INFO:teuthology.orchestra.run.smithi123.stdout:(2/7): cockpit-podman-32-2.module+el8.4.0+11990 2.1 MB/s | 410 kB 00:00 2023-10-07T23:03:51.649 INFO:teuthology.orchestra.run.smithi123.stdout:(3/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 985 kB/s | 192 kB 00:00 2023-10-07T23:03:51.888 INFO:teuthology.orchestra.run.smithi123.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 64 kB/s | 16 kB 00:00 2023-10-07T23:03:51.953 INFO:teuthology.orchestra.run.smithi123.stdout:(5/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 27 MB/s | 8.2 MB 00:00 2023-10-07T23:03:51.992 INFO:teuthology.orchestra.run.smithi123.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 20 MB/s | 7.0 MB 00:00 2023-10-07T23:03:52.031 INFO:teuthology.orchestra.run.smithi123.stdout:(7/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 356 kB/s | 51 kB 00:00 2023-10-07T23:03:52.031 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:03:52.031 INFO:teuthology.orchestra.run.smithi123.stdout:Total 28 MB/s | 16 MB 00:00 2023-10-07T23:03:52.037 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-10-07T23:03:52.054 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-10-07T23:03:52.054 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-10-07T23:03:52.526 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================= 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================= 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout:Installing group/module packages: 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-10-07T23:03:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout:Installing module profiles: 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools/common 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools rhel8 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================= 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout:Install 7 Packages 2023-10-07T23:03:52.576 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:03:52.577 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 16 M 2023-10-07T23:03:52.577 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 57 M 2023-10-07T23:03:52.577 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2023-10-07T23:03:52.600 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-10-07T23:03:52.605 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-10-07T23:03:52.711 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-10-07T23:03:52.728 INFO:teuthology.orchestra.run.smithi086.stdout:(1/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 1.2 MB/s | 192 kB 00:00 2023-10-07T23:03:52.739 INFO:teuthology.orchestra.run.smithi086.stdout:(2/7): yajl-2.1.0-10.el8.x86_64.rpm 251 kB/s | 41 kB 00:00 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================================= 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================================= 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout:Installing group/module packages: 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-10-07T23:03:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout:Installing module profiles: 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: container-tools/common 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling module streams: 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: container-tools rhel8 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================================= 2023-10-07T23:03:52.769 INFO:teuthology.orchestra.run.smithi067.stdout:Install 7 Packages 2023-10-07T23:03:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:03:52.770 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 16 M 2023-10-07T23:03:52.770 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 57 M 2023-10-07T23:03:52.770 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-10-07T23:03:52.778 INFO:teuthology.orchestra.run.smithi086.stdout:(3/7): cockpit-podman-32-2.module+el8.4.0+11990 2.0 MB/s | 410 kB 00:00 2023-10-07T23:03:52.873 INFO:teuthology.orchestra.run.smithi086.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 110 kB/s | 16 kB 00:00 2023-10-07T23:03:52.960 INFO:teuthology.orchestra.run.smithi086.stdout:(5/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 45 MB/s | 8.2 MB 00:00 2023-10-07T23:03:52.974 INFO:teuthology.orchestra.run.smithi086.stdout:(6/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 503 kB/s | 51 kB 00:00 2023-10-07T23:03:52.987 INFO:teuthology.orchestra.run.smithi067.stdout:(1/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 884 kB/s | 192 kB 00:00 2023-10-07T23:03:53.025 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-10-07T23:03:53.041 INFO:teuthology.orchestra.run.smithi067.stdout:(2/7): yajl-2.1.0-10.el8.x86_64.rpm 151 kB/s | 41 kB 00:00 2023-10-07T23:03:53.061 INFO:teuthology.orchestra.run.smithi086.stdout:(7/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 22 MB/s | 7.0 MB 00:00 2023-10-07T23:03:53.062 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:03:53.062 INFO:teuthology.orchestra.run.smithi086.stdout:Total 33 MB/s | 16 MB 00:00 2023-10-07T23:03:53.066 INFO:teuthology.orchestra.run.smithi067.stdout:(3/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 201 kB/s | 16 kB 00:00 2023-10-07T23:03:53.068 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2023-10-07T23:03:53.084 INFO:teuthology.orchestra.run.smithi067.stdout:(4/7): cockpit-podman-32-2.module+el8.4.0+11990 1.3 MB/s | 410 kB 00:00 2023-10-07T23:03:53.085 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2023-10-07T23:03:53.085 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2023-10-07T23:03:53.170 INFO:teuthology.orchestra.run.smithi067.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 594 kB/s | 51 kB 00:00 2023-10-07T23:03:53.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-10-07T23:03:53.247 INFO:teuthology.orchestra.run.smithi067.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 34 MB/s | 7.0 MB 00:00 2023-10-07T23:03:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 28 MB/s | 8.2 MB 00:00 2023-10-07T23:03:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:03:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:Total 27 MB/s | 16 MB 00:00 2023-10-07T23:03:53.368 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-10-07T23:03:53.389 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-10-07T23:03:53.390 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-10-07T23:03:53.457 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-10-07T23:03:53.684 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2023-10-07T23:03:53.689 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2023-10-07T23:03:53.908 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-10-07T23:03:53.913 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-10-07T23:03:54.115 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2023-10-07T23:03:54.277 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-10-07T23:03:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-10-07T23:03:54.406 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-10-07T23:03:54.490 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-10-07T23:03:54.922 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-10-07T23:03:55.148 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-10-07T23:03:55.180 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-10-07T23:03:55.378 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-10-07T23:03:55.383 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-10-07T23:03:55.525 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-10-07T23:03:55.712 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-10-07T23:03:55.972 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-10-07T23:03:56.155 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-10-07T23:03:56.185 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-10-07T23:03:56.333 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-10-07T23:03:56.466 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-10-07T23:03:56.646 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-10-07T23:03:56.908 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-10-07T23:03:57.046 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-10-07T23:03:57.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-10-07T23:03:57.207 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:05:04 ago on Sat 07 Oct 2023 10:58:52 PM UTC. 2023-10-07T23:03:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-10-07T23:03:57.390 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-10-07T23:03:57.588 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-10-07T23:03:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-10-07T23:03:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-10-07T23:03:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-10-07T23:03:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-10-07T23:03:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-10-07T23:03:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-10-07T23:03:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-10-07T23:03:58.204 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-10-07T23:03:58.205 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:05:03 ago on Sat 07 Oct 2023 10:58:54 PM UTC. 2023-10-07T23:03:59.293 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-10-07T23:03:59.293 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:02:11 ago on Sat 07 Oct 2023 11:01:47 PM UTC. 2023-10-07T23:04:00.002 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-10-07T23:04:00.159 INFO:teuthology.orchestra.run.smithi123.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-10-07T23:04:00.160 INFO:teuthology.orchestra.run.smithi123.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-07T23:04:00.160 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:04:00.160 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-10-07T23:04:00.297 DEBUG:teuthology.parallel:result is None 2023-10-07T23:04:00.868 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-10-07T23:04:01.009 INFO:teuthology.orchestra.run.smithi086.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-10-07T23:04:01.010 INFO:teuthology.orchestra.run.smithi086.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-07T23:04:01.010 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:04:01.010 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2023-10-07T23:04:01.142 DEBUG:teuthology.parallel:result is None 2023-10-07T23:04:01.816 INFO:teuthology.orchestra.run.smithi067.stdout:Installed products updated. 2023-10-07T23:04:01.971 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:04:01.971 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-10-07T23:04:01.971 INFO:teuthology.orchestra.run.smithi067.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:04:01.972 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-10-07T23:04:02.185 DEBUG:teuthology.parallel:result is None 2023-10-07T23:04:02.186 INFO:teuthology.run_tasks:Running task install... 2023-10-07T23:04:02.199 DEBUG:teuthology.task.install:project ceph 2023-10-07T23:04:02.199 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-07T23:04:02.200 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-07T23:04:02.200 INFO:teuthology.task.install:Using flavor: default 2023-10-07T23:04:02.205 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-07T23:04:02.205 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-07T23:04:02.206 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-07T23:04:02.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:04:02.208 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-07T23:04:02.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:04:02.210 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-07T23:04:02.210 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:04:02.361 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-07T23:04:02.362 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-07T23:04:02.433 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-07T23:04:02.433 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-07T23:04:02.447 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-07T23:04:02.448 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-07T23:04:02.534 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-07T23:04:02.534 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:04:02.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-07T23:04:02.540 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-07T23:04:02.540 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:04:02.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-07T23:04:02.557 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-07T23:04:02.557 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:04:02.557 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-07T23:04:02.559 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-07T23:04:02.559 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-07T23:04:02.583 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-07T23:04:02.583 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-07T23:04:02.602 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-07T23:04:02.603 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-07T23:04:02.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-07T23:04:02.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-07T23:04:02.695 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-07T23:04:02.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-07T23:04:02.721 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-07T23:04:02.758 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2023-10-07T23:04:02.759 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2023-10-07T23:04:02.783 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2023-10-07T23:04:02.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2023-10-07T23:04:02.795 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-07T23:04:02.856 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2023-10-07T23:04:02.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2023-10-07T23:04:03.155 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2023-10-07T23:04:03.225 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-10-07T23:04:03.359 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-07T23:04:03.629 INFO:teuthology.orchestra.run.smithi123.stdout:44 files removed 2023-10-07T23:04:03.665 INFO:teuthology.orchestra.run.smithi086.stdout:44 files removed 2023-10-07T23:04:03.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-07T23:04:03.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-07T23:04:03.829 INFO:teuthology.orchestra.run.smithi067.stdout:44 files removed 2023-10-07T23:04:03.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-07T23:04:04.096 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2023-10-07T23:04:04.120 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-10-07T23:04:04.385 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-07T23:04:05.076 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 230 kB/s | 78 kB 00:00 2023-10-07T23:04:05.077 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 231 kB/s | 78 kB 00:00 2023-10-07T23:04:05.276 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 274 kB/s | 78 kB 00:00 2023-10-07T23:04:05.364 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-10-07T23:04:05.436 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-07T23:04:05.623 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-07T23:04:05.637 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 6.0 kB/s | 1.5 kB 00:00 2023-10-07T23:04:05.718 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2023-10-07T23:04:05.757 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 38 kB/s | 3.5 kB 00:00 2023-10-07T23:04:05.830 INFO:teuthology.orchestra.run.smithi086.stdout:Copr repo for python3-asyncssh owned by ceph 42 kB/s | 3.5 kB 00:00 2023-10-07T23:04:05.897 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 6.1 kB/s | 1.5 kB 00:00 2023-10-07T23:04:06.010 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-07T23:04:06.405 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2023-10-07T23:04:06.680 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-10-07T23:04:07.865 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2023-10-07T23:04:10.228 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-07T23:04:10.608 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-07T23:04:11.046 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 33 MB 00:00 2023-10-07T23:04:11.354 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 33 MB 00:00 2023-10-07T23:04:11.558 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-07T23:04:12.406 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 33 MB 00:00 2023-10-07T23:04:16.273 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 118 MB/s | 38 MB 00:00 2023-10-07T23:04:16.668 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 122 MB/s | 38 MB 00:00 2023-10-07T23:04:18.480 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 38 MB 00:00 2023-10-07T23:04:24.576 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T23:04:24.576 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T23:04:24.587 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-07T23:04:24.588 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-07T23:04:24.589 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-07T23:04:24.590 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-07T23:04:24.591 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-07T23:04:24.594 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-07T23:04:24.594 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-07T23:04:25.087 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-10-07T23:04:25.098 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================================================== 2023-10-07T23:04:25.098 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-10-07T23:04:25.098 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================================================== 2023-10-07T23:04:25.098 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-10-07T23:04:25.098 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T23:04:25.099 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-07T23:04:25.100 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-07T23:04:25.101 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-07T23:04:25.102 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-07T23:04:25.103 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-07T23:04:25.104 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-07T23:04:25.105 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-07T23:04:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-07T23:04:25.107 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-07T23:04:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-07T23:04:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-07T23:04:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================================================== 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout:Install 118 Packages 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 202 M 2023-10-07T23:04:25.111 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-10-07T23:04:25.397 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T23:04:25.397 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T23:04:25.409 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-07T23:04:25.410 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-07T23:04:25.411 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-07T23:04:25.412 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-07T23:04:25.413 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-07T23:04:25.416 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-07T23:04:25.417 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-07T23:04:25.461 INFO:teuthology.orchestra.run.smithi123.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 40 kB/s | 6.4 kB 00:00 2023-10-07T23:04:25.855 INFO:teuthology.orchestra.run.smithi123.stdout:(2/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 9.3 MB/s | 5.2 MB 00:00 2023-10-07T23:04:25.863 INFO:teuthology.orchestra.run.smithi123.stdout:(3/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 2.1 MB/s | 876 kB 00:00 2023-10-07T23:04:25.886 INFO:teuthology.orchestra.run.smithi123.stdout:(4/120): ceph-immutable-object-cache-17.2.6-132 5.3 MB/s | 162 kB 00:00 2023-10-07T23:04:25.914 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2023-10-07T23:04:25.925 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================================================== 2023-10-07T23:04:25.925 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2023-10-07T23:04:25.925 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================================================== 2023-10-07T23:04:25.925 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2023-10-07T23:04:25.925 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-07T23:04:25.925 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-07T23:04:25.926 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-07T23:04:25.927 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T23:04:25.928 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-07T23:04:25.929 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-07T23:04:25.930 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-07T23:04:25.931 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-07T23:04:25.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-07T23:04:25.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-07T23:04:25.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-07T23:04:25.937 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-07T23:04:25.938 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-07T23:04:25.939 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout:====================================================================================================================== 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout:Install 118 Packages 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 2 Packages 2023-10-07T23:04:25.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:04:25.941 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 202 M 2023-10-07T23:04:25.941 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2023-10-07T23:04:26.031 INFO:teuthology.orchestra.run.smithi123.stdout:(5/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 11 MB/s | 1.6 MB 00:00 2023-10-07T23:04:26.385 INFO:teuthology.orchestra.run.smithi123.stdout:(6/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 11 MB/s | 3.9 MB 00:00 2023-10-07T23:04:26.605 INFO:teuthology.orchestra.run.smithi086.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-10-07T23:04:26.662 INFO:teuthology.orchestra.run.smithi123.stdout:(7/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-10-07T23:04:27.158 INFO:teuthology.orchestra.run.smithi123.stdout:(8/120): ceph-common-17.2.6-1320.g08cac018.el8. 13 MB/s | 23 MB 00:01 2023-10-07T23:04:27.190 INFO:teuthology.orchestra.run.smithi123.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 767 kB/s | 24 kB 00:00 2023-10-07T23:04:27.733 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T23:04:27.734 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-07T23:04:27.745 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-07T23:04:27.746 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-07T23:04:27.747 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-07T23:04:27.748 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-07T23:04:27.750 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-07T23:04:27.752 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-07T23:04:27.753 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-07T23:04:27.944 INFO:teuthology.orchestra.run.smithi123.stdout:(10/120): ceph-osd-17.2.6-1320.g08cac018.el8.x8 11 MB/s | 17 MB 00:01 2023-10-07T23:04:27.998 INFO:teuthology.orchestra.run.smithi123.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 4.7 MB/s | 255 kB 00:00 2023-10-07T23:04:28.047 INFO:teuthology.orchestra.run.smithi123.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 505 kB/s | 25 kB 00:00 2023-10-07T23:04:28.163 INFO:teuthology.orchestra.run.smithi123.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 6.7 MB/s | 797 kB 00:00 2023-10-07T23:04:28.215 INFO:teuthology.orchestra.run.smithi123.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 3.3 MB/s | 174 kB 00:00 2023-10-07T23:04:28.268 INFO:teuthology.orchestra.run.smithi123.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 2.4 MB/s | 130 kB 00:00 2023-10-07T23:04:28.282 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-10-07T23:04:28.295 INFO:teuthology.orchestra.run.smithi067.stdout:====================================================================================================================== 2023-10-07T23:04:28.295 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-10-07T23:04:28.295 INFO:teuthology.orchestra.run.smithi067.stdout:====================================================================================================================== 2023-10-07T23:04:28.295 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-10-07T23:04:28.295 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-07T23:04:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-07T23:04:28.297 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-07T23:04:28.298 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-07T23:04:28.299 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-07T23:04:28.300 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-07T23:04:28.301 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-07T23:04:28.302 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-07T23:04:28.303 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-07T23:04:28.304 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-07T23:04:28.305 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout:====================================================================================================================== 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout:Install 118 Packages 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 2 Packages 2023-10-07T23:04:28.306 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:04:28.307 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 202 M 2023-10-07T23:04:28.307 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-10-07T23:04:28.331 INFO:teuthology.orchestra.run.smithi123.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 7.9 MB/s | 507 kB 00:00 2023-10-07T23:04:28.333 INFO:teuthology.orchestra.run.smithi086.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 507 kB/s | 876 kB 00:01 2023-10-07T23:04:28.710 INFO:teuthology.orchestra.run.smithi086.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 429 kB/s | 162 kB 00:00 2023-10-07T23:04:28.750 INFO:teuthology.orchestra.run.smithi067.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-10-07T23:04:28.940 INFO:teuthology.orchestra.run.smithi123.stdout:(17/120): librgw2-17.2.6-1320.g08cac018.el8.x86 9.1 MB/s | 5.6 MB 00:00 2023-10-07T23:04:28.997 INFO:teuthology.orchestra.run.smithi123.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 787 kB/s | 45 kB 00:00 2023-10-07T23:04:29.065 INFO:teuthology.orchestra.run.smithi123.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 1.7 MB/s | 115 kB 00:00 2023-10-07T23:04:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 2.4 MB/s | 190 kB 00:00 2023-10-07T23:04:29.190 INFO:teuthology.orchestra.run.smithi067.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 1.9 MB/s | 876 kB 00:00 2023-10-07T23:04:29.264 INFO:teuthology.orchestra.run.smithi067.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 2.1 MB/s | 162 kB 00:00 2023-10-07T23:04:29.572 INFO:teuthology.orchestra.run.smithi123.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 865 kB/s | 371 kB 00:00 2023-10-07T23:04:29.797 INFO:teuthology.orchestra.run.smithi067.stdout:(4/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 4.3 MB/s | 2.3 MB 00:00 2023-10-07T23:04:30.038 INFO:teuthology.orchestra.run.smithi123.stdout:(22/120): python3-rbd-17.2.6-1320.g08cac018.el8 768 kB/s | 358 kB 00:00 2023-10-07T23:04:30.096 INFO:teuthology.orchestra.run.smithi123.stdout:(23/120): python3-rgw-17.2.6-1320.g08cac018.el8 1.8 MB/s | 107 kB 00:00 2023-10-07T23:04:30.154 INFO:teuthology.orchestra.run.smithi123.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.5 MB/s | 90 kB 00:00 2023-10-07T23:04:30.164 INFO:teuthology.orchestra.run.smithi067.stdout:(5/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 4.3 MB/s | 1.6 MB 00:00 2023-10-07T23:04:30.575 INFO:teuthology.orchestra.run.smithi067.stdout:(6/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 2.4 MB/s | 5.2 MB 00:02 2023-10-07T23:04:30.952 INFO:teuthology.orchestra.run.smithi123.stdout:(25/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 4.3 MB/s | 3.5 MB 00:00 2023-10-07T23:04:31.020 INFO:teuthology.orchestra.run.smithi123.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 2.5 MB/s | 173 kB 00:00 2023-10-07T23:04:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 360 kB/s | 23 kB 00:00 2023-10-07T23:04:31.102 INFO:teuthology.orchestra.run.smithi067.stdout:(7/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 4.2 MB/s | 3.9 MB 00:00 2023-10-07T23:04:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 2.1 MB/s | 121 kB 00:00 2023-10-07T23:04:31.578 INFO:teuthology.orchestra.run.smithi123.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 3.9 MB/s | 1.7 MB 00:00 2023-10-07T23:04:31.663 INFO:teuthology.orchestra.run.smithi086.stdout:(4/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 982 kB/s | 5.2 MB 00:05 2023-10-07T23:04:32.495 INFO:teuthology.orchestra.run.smithi086.stdout:(5/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 624 kB/s | 2.3 MB 00:03 2023-10-07T23:04:32.588 INFO:teuthology.orchestra.run.smithi067.stdout:(8/120): ceph-common-17.2.6-1320.g08cac018.el8. 5.6 MB/s | 23 MB 00:04 2023-10-07T23:04:32.636 INFO:teuthology.orchestra.run.smithi067.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 511 kB/s | 24 kB 00:00 2023-10-07T23:04:32.641 INFO:teuthology.orchestra.run.smithi086.stdout:(6/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 1.6 MB/s | 1.6 MB 00:00 2023-10-07T23:04:32.662 INFO:teuthology.orchestra.run.smithi123.stdout:(30/120): ceph-test-17.2.6-1320.g08cac018.el8.x 9.4 MB/s | 52 MB 00:05 2023-10-07T23:04:32.826 INFO:teuthology.orchestra.run.smithi123.stdout:(31/120): ceph-radosgw-17.2.6-1320.g08cac018.el 2.7 MB/s | 17 MB 00:06 2023-10-07T23:04:32.828 INFO:teuthology.orchestra.run.smithi123.stdout:(32/120): ceph-mgr-modules-core-17.2.6-1320.g08 1.4 MB/s | 238 kB 00:00 2023-10-07T23:04:32.889 INFO:teuthology.orchestra.run.smithi123.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 240 kB/s | 14 kB 00:00 2023-10-07T23:04:32.900 INFO:teuthology.orchestra.run.smithi123.stdout:(34/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 642 kB/s | 47 kB 00:00 2023-10-07T23:04:32.955 INFO:teuthology.orchestra.run.smithi123.stdout:(35/120): ceph-volume-17.2.6-1320.g08cac018.el8 4.0 MB/s | 267 kB 00:00 2023-10-07T23:04:32.987 INFO:teuthology.orchestra.run.smithi123.stdout:(36/120): cephadm-17.2.6-1320.g08cac018.el8.noa 932 kB/s | 80 kB 00:00 2023-10-07T23:04:33.016 INFO:teuthology.orchestra.run.smithi123.stdout:(37/120): cephfs-top-17.2.6-1320.g08cac018.el8. 294 kB/s | 18 kB 00:00 2023-10-07T23:04:33.082 INFO:teuthology.orchestra.run.smithi123.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 898 kB/s | 59 kB 00:00 2023-10-07T23:04:33.102 INFO:teuthology.orchestra.run.smithi123.stdout:(39/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.6 MB/s | 306 kB 00:00 2023-10-07T23:04:33.104 INFO:teuthology.orchestra.run.smithi123.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 3.3 MB/s | 75 kB 00:00 2023-10-07T23:04:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 3.4 MB/s | 44 kB 00:00 2023-10-07T23:04:33.127 INFO:teuthology.orchestra.run.smithi123.stdout:(42/120): python3-asyncssh-2.7.0-2.el8.noarch.r 19 MB/s | 476 kB 00:00 2023-10-07T23:04:33.138 INFO:teuthology.orchestra.run.smithi123.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 2.9 MB/s | 33 kB 00:00 2023-10-07T23:04:33.140 INFO:teuthology.orchestra.run.smithi123.stdout:(44/120): python3-beautifulsoup4-4.6.3-2.el8.1. 7.8 MB/s | 185 kB 00:00 2023-10-07T23:04:33.150 INFO:teuthology.orchestra.run.smithi123.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 1.4 MB/s | 16 kB 00:00 2023-10-07T23:04:33.154 INFO:teuthology.orchestra.run.smithi123.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 13 MB/s | 173 kB 00:00 2023-10-07T23:04:33.166 INFO:teuthology.orchestra.run.smithi123.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 24 MB/s | 384 kB 00:00 2023-10-07T23:04:33.167 INFO:teuthology.orchestra.run.smithi123.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 6.3 MB/s | 81 kB 00:00 2023-10-07T23:04:33.178 INFO:teuthology.orchestra.run.smithi123.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 984 kB/s | 11 kB 00:00 2023-10-07T23:04:33.178 INFO:teuthology.orchestra.run.smithi123.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 1.5 MB/s | 18 kB 00:00 2023-10-07T23:04:33.192 INFO:teuthology.orchestra.run.smithi123.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 3.7 MB/s | 49 kB 00:00 2023-10-07T23:04:33.223 INFO:teuthology.orchestra.run.smithi123.stdout:(52/120): ceph-mgr-diskprediction-local-17.2.6- 4.5 MB/s | 7.4 MB 00:01 2023-10-07T23:04:33.224 INFO:teuthology.orchestra.run.smithi123.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.8 MB/s | 59 kB 00:00 2023-10-07T23:04:33.228 INFO:teuthology.orchestra.run.smithi123.stdout:(54/120): python3-kubernetes-11.0.0-6.el8.noarc 23 MB/s | 1.2 MB 00:00 2023-10-07T23:04:33.241 INFO:teuthology.orchestra.run.smithi123.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 1.3 MB/s | 16 kB 00:00 2023-10-07T23:04:33.243 INFO:teuthology.orchestra.run.smithi123.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 15 MB/s | 283 kB 00:00 2023-10-07T23:04:33.253 INFO:teuthology.orchestra.run.smithi123.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 2.9 MB/s | 34 kB 00:00 2023-10-07T23:04:33.257 INFO:teuthology.orchestra.run.smithi123.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 14 MB/s | 196 kB 00:00 2023-10-07T23:04:33.265 INFO:teuthology.orchestra.run.smithi123.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 5.2 MB/s | 64 kB 00:00 2023-10-07T23:04:33.268 INFO:teuthology.orchestra.run.smithi123.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 1.6 MB/s | 19 kB 00:00 2023-10-07T23:04:33.276 INFO:teuthology.orchestra.run.smithi123.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 2.1 MB/s | 24 kB 00:00 2023-10-07T23:04:33.280 INFO:teuthology.orchestra.run.smithi123.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 2.5 MB/s | 29 kB 00:00 2023-10-07T23:04:33.288 INFO:teuthology.orchestra.run.smithi123.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 2.4 MB/s | 27 kB 00:00 2023-10-07T23:04:33.289 INFO:teuthology.orchestra.run.smithi123.stdout:(64/120): python3-natsort-7.1.1-2.el8.noarch.rp 915 kB/s | 60 kB 00:00 2023-10-07T23:04:33.295 INFO:teuthology.orchestra.run.smithi123.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 17 MB/s | 255 kB 00:00 2023-10-07T23:04:33.303 INFO:teuthology.orchestra.run.smithi123.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 17 MB/s | 251 kB 00:00 2023-10-07T23:04:33.303 INFO:teuthology.orchestra.run.smithi123.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 4.3 MB/s | 61 kB 00:00 2023-10-07T23:04:33.307 INFO:teuthology.orchestra.run.smithi123.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 6.8 MB/s | 85 kB 00:00 2023-10-07T23:04:33.312 INFO:teuthology.orchestra.run.smithi123.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-10-07T23:04:33.314 INFO:teuthology.orchestra.run.smithi123.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 2.0 MB/s | 23 kB 00:00 2023-10-07T23:04:33.369 INFO:teuthology.orchestra.run.smithi123.stdout:(71/120): thrift-0.13.0-2.el8.x86_64.rpm 27 MB/s | 1.7 MB 00:00 2023-10-07T23:04:33.426 INFO:teuthology.orchestra.run.smithi123.stdout:(72/120): python3-prettytable-0.7.2-14.el8.noar 396 kB/s | 44 kB 00:00 2023-10-07T23:04:33.480 INFO:teuthology.orchestra.run.smithi123.stdout:(73/120): python3-jsonpatch-1.21-2.el8.noarch.r 239 kB/s | 27 kB 00:00 2023-10-07T23:04:33.546 INFO:teuthology.orchestra.run.smithi123.stdout:(74/120): jq-1.5-12.el8.x86_64.rpm 1.3 MB/s | 161 kB 00:00 2023-10-07T23:04:33.584 INFO:teuthology.orchestra.run.smithi123.stdout:(75/120): flex-2.6.1-9.el8.x86_64.rpm 3.0 MB/s | 320 kB 00:00 2023-10-07T23:04:33.707 INFO:teuthology.orchestra.run.smithi123.stdout:(76/120): python3-babel-2.5.1-5.el8.noarch.rpm 39 MB/s | 4.8 MB 00:00 2023-10-07T23:04:33.818 INFO:teuthology.orchestra.run.smithi123.stdout:(77/120): python3-scipy-1.0.0-20.module+el8.1.0 29 MB/s | 15 MB 00:00 2023-10-07T23:04:33.821 INFO:teuthology.orchestra.run.smithi123.stdout:(78/120): python3-werkzeug-0.12.2-4.el8.noarch. 1.6 MB/s | 457 kB 00:00 2023-10-07T23:04:33.842 INFO:teuthology.orchestra.run.smithi123.stdout:(79/120): python3-markupsafe-0.23-19.el8.x86_64 289 kB/s | 39 kB 00:00 2023-10-07T23:04:33.864 INFO:teuthology.orchestra.run.smithi123.stdout:(80/120): bison-3.0.4-10.el8.x86_64.rpm 15 MB/s | 688 kB 00:00 2023-10-07T23:04:33.887 INFO:teuthology.orchestra.run.smithi123.stdout:(81/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 5.4 MB/s | 126 kB 00:00 2023-10-07T23:04:33.922 INFO:teuthology.orchestra.run.smithi123.stdout:(82/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.6 MB/s | 259 kB 00:00 2023-10-07T23:04:33.945 INFO:teuthology.orchestra.run.smithi123.stdout:(83/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 2.1 MB/s | 225 kB 00:00 2023-10-07T23:04:33.967 INFO:teuthology.orchestra.run.smithi123.stdout:(84/120): oniguruma-6.8.2-2.el8.x86_64.rpm 3.9 MB/s | 187 kB 00:00 2023-10-07T23:04:34.019 INFO:teuthology.orchestra.run.smithi067.stdout:(10/120): ceph-radosgw-17.2.6-1320.g08cac018.el 5.8 MB/s | 17 MB 00:02 2023-10-07T23:04:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 26 MB/s | 3.7 MB 00:00 2023-10-07T23:04:34.038 INFO:teuthology.orchestra.run.smithi123.stdout:(86/120): python3-jinja2-2.10.1-2.el8_0.noarch. 5.7 MB/s | 538 kB 00:00 2023-10-07T23:04:34.068 INFO:teuthology.orchestra.run.smithi067.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 5.1 MB/s | 255 kB 00:00 2023-10-07T23:04:34.077 INFO:teuthology.orchestra.run.smithi123.stdout:(87/120): socat-1.7.3.3-2.el8.x86_64.rpm 2.7 MB/s | 302 kB 00:00 2023-10-07T23:04:34.084 INFO:teuthology.orchestra.run.smithi123.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 2.4 MB/s | 110 kB 00:00 2023-10-07T23:04:34.108 INFO:teuthology.orchestra.run.smithi067.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 623 kB/s | 25 kB 00:00 2023-10-07T23:04:34.118 INFO:teuthology.orchestra.run.smithi123.stdout:(89/120): python3-jsonpointer-1.10-11.el8.noarc 586 kB/s | 20 kB 00:00 2023-10-07T23:04:34.121 INFO:teuthology.orchestra.run.smithi123.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 7.8 MB/s | 353 kB 00:00 2023-10-07T23:04:34.160 INFO:teuthology.orchestra.run.smithi123.stdout:(91/120): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-10-07T23:04:34.164 INFO:teuthology.orchestra.run.smithi123.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-10-07T23:04:34.222 INFO:teuthology.orchestra.run.smithi123.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 6.7 MB/s | 709 kB 00:00 2023-10-07T23:04:34.239 INFO:teuthology.orchestra.run.smithi067.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 6.0 MB/s | 797 kB 00:00 2023-10-07T23:04:34.273 INFO:teuthology.orchestra.run.smithi123.stdout:(94/120): openblas-threads-0.3.12-1.el8.x86_64. 42 MB/s | 4.7 MB 00:00 2023-10-07T23:04:34.279 INFO:teuthology.orchestra.run.smithi067.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 4.3 MB/s | 174 kB 00:00 2023-10-07T23:04:34.321 INFO:teuthology.orchestra.run.smithi067.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 3.0 MB/s | 130 kB 00:00 2023-10-07T23:04:34.322 INFO:teuthology.orchestra.run.smithi123.stdout:(95/120): openblas-0.3.12-1.el8.x86_64.rpm 29 MB/s | 4.6 MB 00:00 2023-10-07T23:04:34.376 INFO:teuthology.orchestra.run.smithi123.stdout:(96/120): automake-1.16.1-7.el8.noarch.rpm 4.5 MB/s | 713 kB 00:00 2023-10-07T23:04:34.397 INFO:teuthology.orchestra.run.smithi123.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 570 kB/s | 43 kB 00:00 2023-10-07T23:04:34.411 INFO:teuthology.orchestra.run.smithi067.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 5.5 MB/s | 507 kB 00:00 2023-10-07T23:04:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:(98/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 104 kB/s | 19 kB 00:00 2023-10-07T23:04:34.520 INFO:teuthology.orchestra.run.smithi123.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-07T23:04:34.546 INFO:teuthology.orchestra.run.smithi123.stdout:(100/120): python3-oauthlib-2.1.0-1.el8.noarch. 912 kB/s | 155 kB 00:00 2023-10-07T23:04:34.550 INFO:teuthology.orchestra.run.smithi123.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 488 kB/s | 48 kB 00:00 2023-10-07T23:04:34.574 INFO:teuthology.orchestra.run.smithi123.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.2 MB/s | 229 kB 00:00 2023-10-07T23:04:34.583 INFO:teuthology.orchestra.run.smithi067.stdout:(17/120): ceph-osd-17.2.6-1320.g08cac018.el8.x8 4.2 MB/s | 17 MB 00:04 2023-10-07T23:04:34.640 INFO:teuthology.orchestra.run.smithi067.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 786 kB/s | 45 kB 00:00 2023-10-07T23:04:34.641 INFO:teuthology.orchestra.run.smithi123.stdout:(103/120): python3-requests-oauthlib-1.0.0-1.el 447 kB/s | 43 kB 00:00 2023-10-07T23:04:34.666 INFO:teuthology.orchestra.run.smithi123.stdout:(104/120): openssl-devel-1.1.1g-15.el8_3.x86_64 20 MB/s | 2.3 MB 00:00 2023-10-07T23:04:34.668 INFO:teuthology.orchestra.run.smithi123.stdout:(105/120): libverto-devel-0.3.0-5.el8.x86_64.rp 189 kB/s | 18 kB 00:00 2023-10-07T23:04:34.701 INFO:teuthology.orchestra.run.smithi067.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 1.9 MB/s | 115 kB 00:00 2023-10-07T23:04:34.714 INFO:teuthology.orchestra.run.smithi123.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 1.7 MB/s | 83 kB 00:00 2023-10-07T23:04:34.726 INFO:teuthology.orchestra.run.smithi123.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 1.0 MB/s | 59 kB 00:00 2023-10-07T23:04:34.737 INFO:teuthology.orchestra.run.smithi123.stdout:(108/120): pcre2-devel-10.32-2.el8.x86_64.rpm 6.2 MB/s | 605 kB 00:00 2023-10-07T23:04:34.764 INFO:teuthology.orchestra.run.smithi067.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 3.0 MB/s | 190 kB 00:00 2023-10-07T23:04:34.851 INFO:teuthology.orchestra.run.smithi123.stdout:(109/120): libsepol-devel-2.9-2.el8.x86_64.rpm 755 kB/s | 86 kB 00:00 2023-10-07T23:04:34.852 INFO:teuthology.orchestra.run.smithi123.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 301 kB/s | 38 kB 00:00 2023-10-07T23:04:34.868 INFO:teuthology.orchestra.run.smithi123.stdout:(111/120): libquadmath-8.4.1-1.el8.x86_64.rpm 1.1 MB/s | 169 kB 00:00 2023-10-07T23:04:34.880 INFO:teuthology.orchestra.run.smithi067.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 3.1 MB/s | 371 kB 00:00 2023-10-07T23:04:34.956 INFO:teuthology.orchestra.run.smithi123.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 451 kB/s | 47 kB 00:00 2023-10-07T23:04:34.976 INFO:teuthology.orchestra.run.smithi123.stdout:(113/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 462 kB/s | 58 kB 00:00 2023-10-07T23:04:34.996 INFO:teuthology.orchestra.run.smithi067.stdout:(22/120): python3-rbd-17.2.6-1320.g08cac018.el8 3.0 MB/s | 358 kB 00:00 2023-10-07T23:04:35.017 INFO:teuthology.orchestra.run.smithi123.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 3.7 MB/s | 559 kB 00:00 2023-10-07T23:04:35.032 INFO:teuthology.orchestra.run.smithi123.stdout:(115/120): libgfortran-8.4.1-1.el8.x86_64.rpm 8.3 MB/s | 639 kB 00:00 2023-10-07T23:04:35.052 INFO:teuthology.orchestra.run.smithi123.stdout:(116/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 924 kB/s | 70 kB 00:00 2023-10-07T23:04:35.058 INFO:teuthology.orchestra.run.smithi067.stdout:(23/120): python3-rgw-17.2.6-1320.g08cac018.el8 1.7 MB/s | 107 kB 00:00 2023-10-07T23:04:35.118 INFO:teuthology.orchestra.run.smithi123.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 1.9 MB/s | 200 kB 00:00 2023-10-07T23:04:35.130 INFO:teuthology.orchestra.run.smithi067.stdout:(24/120): librgw2-17.2.6-1320.g08cac018.el8.x86 7.7 MB/s | 5.6 MB 00:00 2023-10-07T23:04:35.133 INFO:teuthology.orchestra.run.smithi067.stdout:(25/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.2 MB/s | 90 kB 00:00 2023-10-07T23:04:35.160 INFO:teuthology.orchestra.run.smithi123.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 760 kB/s | 97 kB 00:00 2023-10-07T23:04:35.197 INFO:teuthology.orchestra.run.smithi067.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 2.6 MB/s | 173 kB 00:00 2023-10-07T23:04:35.257 INFO:teuthology.orchestra.run.smithi067.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 389 kB/s | 23 kB 00:00 2023-10-07T23:04:35.323 INFO:teuthology.orchestra.run.smithi067.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 1.8 MB/s | 121 kB 00:00 2023-10-07T23:04:35.656 INFO:teuthology.orchestra.run.smithi086.stdout:(7/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 1.2 MB/s | 3.9 MB 00:03 2023-10-07T23:04:35.707 INFO:teuthology.orchestra.run.smithi067.stdout:(29/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 6.0 MB/s | 3.5 MB 00:00 2023-10-07T23:04:35.781 INFO:teuthology.orchestra.run.smithi067.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 3.7 MB/s | 1.7 MB 00:00 2023-10-07T23:04:35.868 INFO:teuthology.orchestra.run.smithi067.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 2.7 MB/s | 238 kB 00:00 2023-10-07T23:04:35.946 INFO:teuthology.orchestra.run.smithi067.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 606 kB/s | 47 kB 00:00 2023-10-07T23:04:36.022 INFO:teuthology.orchestra.run.smithi067.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 191 kB/s | 14 kB 00:00 2023-10-07T23:04:36.054 INFO:teuthology.orchestra.run.smithi123.stdout:(119/120): librbd1-17.2.6-1320.g08cac018.el8.x8 3.8 MB/s | 3.6 MB 00:00 2023-10-07T23:04:36.104 INFO:teuthology.orchestra.run.smithi067.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 3.2 MB/s | 267 kB 00:00 2023-10-07T23:04:36.169 INFO:teuthology.orchestra.run.smithi067.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 1.2 MB/s | 80 kB 00:00 2023-10-07T23:04:36.234 INFO:teuthology.orchestra.run.smithi067.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 280 kB/s | 18 kB 00:00 2023-10-07T23:04:36.368 INFO:teuthology.orchestra.run.smithi123.stdout:(120/120): librados2-17.2.6-1320.g08cac018.el8. 2.6 MB/s | 3.5 MB 00:01 2023-10-07T23:04:36.375 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:04:36.376 INFO:teuthology.orchestra.run.smithi123.stdout:Total 18 MB/s | 202 MB 00:11 2023-10-07T23:04:36.426 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-10-07T23:04:36.541 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-10-07T23:04:36.541 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-10-07T23:04:36.728 INFO:teuthology.orchestra.run.smithi067.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 619 kB/s | 306 kB 00:00 2023-10-07T23:04:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:(38/120): ceph-mgr-diskprediction-local-17.2.6- 6.9 MB/s | 7.4 MB 00:01 2023-10-07T23:04:36.809 INFO:teuthology.orchestra.run.smithi067.stdout:(39/120): liboath-2.6.2-3.el8.x86_64.rpm 731 kB/s | 59 kB 00:00 2023-10-07T23:04:36.970 INFO:teuthology.orchestra.run.smithi067.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-10-07T23:04:37.044 INFO:teuthology.orchestra.run.smithi067.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 588 kB/s | 44 kB 00:00 2023-10-07T23:04:37.117 INFO:teuthology.orchestra.run.smithi067.stdout:(42/120): libunwind-1.3.1-3.el8.x86_64.rpm 222 kB/s | 75 kB 00:00 2023-10-07T23:04:37.128 INFO:teuthology.orchestra.run.smithi067.stdout:(43/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-10-07T23:04:37.307 INFO:teuthology.orchestra.run.smithi067.stdout:(44/120): ceph-test-17.2.6-1320.g08cac018.el8.x 11 MB/s | 52 MB 00:04 2023-10-07T23:04:37.308 INFO:teuthology.orchestra.run.smithi067.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 91 kB/s | 16 kB 00:00 2023-10-07T23:04:37.308 INFO:teuthology.orchestra.run.smithi067.stdout:(46/120): python3-cachetools-3.1.1-4.el8.noarch 172 kB/s | 33 kB 00:00 2023-10-07T23:04:37.406 INFO:teuthology.orchestra.run.smithi067.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.9 MB/s | 384 kB 00:00 2023-10-07T23:04:37.443 INFO:teuthology.orchestra.run.smithi067.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 605 kB/s | 81 kB 00:00 2023-10-07T23:04:37.460 INFO:teuthology.orchestra.run.smithi086.stdout:(8/120): ceph-osd-17.2.6-1320.g08cac018.el8.x86 3.5 MB/s | 17 MB 00:04 2023-10-07T23:04:37.478 INFO:teuthology.orchestra.run.smithi067.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 154 kB/s | 11 kB 00:00 2023-10-07T23:04:37.482 INFO:teuthology.orchestra.run.smithi086.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 1.1 MB/s | 24 kB 00:00 2023-10-07T23:04:37.523 INFO:teuthology.orchestra.run.smithi067.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 220 kB/s | 18 kB 00:00 2023-10-07T23:04:37.622 INFO:teuthology.orchestra.run.smithi067.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 494 kB/s | 49 kB 00:00 2023-10-07T23:04:37.632 INFO:teuthology.orchestra.run.smithi067.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 7.6 MB/s | 1.2 MB 00:00 2023-10-07T23:04:37.705 INFO:teuthology.orchestra.run.smithi067.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 831 kB/s | 60 kB 00:00 2023-10-07T23:04:37.725 INFO:teuthology.orchestra.run.smithi067.stdout:(54/120): python3-more-itertools-7.2.0-3.el8.no 570 kB/s | 59 kB 00:00 2023-10-07T23:04:37.740 INFO:teuthology.orchestra.run.smithi067.stdout:(55/120): python3-cheroot-8.5.2-1.el8.noarch.rp 399 kB/s | 173 kB 00:00 2023-10-07T23:04:37.783 INFO:teuthology.orchestra.run.smithi067.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.5 MB/s | 283 kB 00:00 2023-10-07T23:04:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:(57/120): python3-portend-2.6-1.el8.noarch.rpm 199 kB/s | 16 kB 00:00 2023-10-07T23:04:37.821 INFO:teuthology.orchestra.run.smithi067.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 416 kB/s | 34 kB 00:00 2023-10-07T23:04:37.860 INFO:teuthology.orchestra.run.smithi067.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-10-07T23:04:37.898 INFO:teuthology.orchestra.run.smithi067.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 246 kB/s | 19 kB 00:00 2023-10-07T23:04:37.910 INFO:teuthology.orchestra.run.smithi067.stdout:(61/120): python3-rsa-4.9-2.el8.noarch.rpm 605 kB/s | 64 kB 00:00 2023-10-07T23:04:37.932 INFO:teuthology.orchestra.run.smithi067.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 331 kB/s | 24 kB 00:00 2023-10-07T23:04:37.977 INFO:teuthology.orchestra.run.smithi067.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 368 kB/s | 29 kB 00:00 2023-10-07T23:04:37.997 INFO:teuthology.orchestra.run.smithi067.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 320 kB/s | 27 kB 00:00 2023-10-07T23:04:38.010 INFO:teuthology.orchestra.run.smithi067.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 3.2 MB/s | 255 kB 00:00 2023-10-07T23:04:38.057 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-10-07T23:04:38.062 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-10-07T23:04:38.084 INFO:teuthology.orchestra.run.smithi067.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-07T23:04:38.115 INFO:teuthology.orchestra.run.smithi067.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 519 kB/s | 61 kB 00:00 2023-10-07T23:04:38.126 INFO:teuthology.orchestra.run.smithi067.stdout:(68/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2023-10-07T23:04:38.132 INFO:teuthology.orchestra.run.smithi067.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 14 MB/s | 70 kB 00:00 2023-10-07T23:04:38.157 INFO:teuthology.orchestra.run.smithi067.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 317 kB/s | 23 kB 00:00 2023-10-07T23:04:38.257 INFO:teuthology.orchestra.run.smithi067.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 438 kB/s | 44 kB 00:00 2023-10-07T23:04:38.334 INFO:teuthology.orchestra.run.smithi067.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 346 kB/s | 27 kB 00:00 2023-10-07T23:04:38.432 INFO:teuthology.orchestra.run.smithi067.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 1.6 MB/s | 161 kB 00:00 2023-10-07T23:04:38.539 INFO:teuthology.orchestra.run.smithi067.stdout:(74/120): python3-scipy-1.0.0-20.module+el8.1.0 36 MB/s | 15 MB 00:00 2023-10-07T23:04:38.547 INFO:teuthology.orchestra.run.smithi067.stdout:(75/120): thrift-0.13.0-2.el8.x86_64.rpm 4.0 MB/s | 1.7 MB 00:00 2023-10-07T23:04:38.553 INFO:teuthology.orchestra.run.smithi067.stdout:(76/120): flex-2.6.1-9.el8.x86_64.rpm 2.6 MB/s | 320 kB 00:00 2023-10-07T23:04:38.626 INFO:teuthology.orchestra.run.smithi067.stdout:(77/120): python3-werkzeug-0.12.2-4.el8.noarch. 5.2 MB/s | 457 kB 00:00 2023-10-07T23:04:38.656 INFO:teuthology.orchestra.run.smithi067.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 23 MB/s | 688 kB 00:00 2023-10-07T23:04:38.731 INFO:teuthology.orchestra.run.smithi067.stdout:(79/120): python3-markupsafe-0.23-19.el8.x86_64 217 kB/s | 39 kB 00:00 2023-10-07T23:04:38.752 INFO:teuthology.orchestra.run.smithi067.stdout:(80/120): python3-babel-2.5.1-5.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-10-07T23:04:38.825 INFO:teuthology.orchestra.run.smithi067.stdout:(81/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2023-10-07T23:04:38.868 INFO:teuthology.orchestra.run.smithi067.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-10-07T23:04:38.883 INFO:teuthology.orchestra.run.smithi067.stdout:(83/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 1.5 MB/s | 225 kB 00:00 2023-10-07T23:04:39.009 INFO:teuthology.orchestra.run.smithi067.stdout:(84/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 20 MB/s | 3.7 MB 00:00 2023-10-07T23:04:39.012 INFO:teuthology.orchestra.run.smithi067.stdout:(85/120): oniguruma-6.8.2-2.el8.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-10-07T23:04:39.043 INFO:teuthology.orchestra.run.smithi067.stdout:(86/120): python3-jinja2-2.10.1-2.el8_0.noarch. 3.3 MB/s | 538 kB 00:00 2023-10-07T23:04:39.138 INFO:teuthology.orchestra.run.smithi067.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-10-07T23:04:39.142 INFO:teuthology.orchestra.run.smithi067.stdout:(88/120): socat-1.7.3.3-2.el8.x86_64.rpm 2.2 MB/s | 302 kB 00:00 2023-10-07T23:04:39.169 INFO:teuthology.orchestra.run.smithi067.stdout:(89/120): python3-pyasn1-modules-0.3.7-6.el8.no 875 kB/s | 110 kB 00:00 2023-10-07T23:04:39.276 INFO:teuthology.orchestra.run.smithi067.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 149 kB/s | 20 kB 00:00 2023-10-07T23:04:39.284 INFO:teuthology.orchestra.run.smithi067.stdout:(91/120): librdkafka-0.11.4-1.el8.x86_64.rpm 2.4 MB/s | 353 kB 00:00 2023-10-07T23:04:39.332 INFO:teuthology.orchestra.run.smithi067.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 4.2 MB/s | 709 kB 00:00 2023-10-07T23:04:39.412 INFO:teuthology.orchestra.run.smithi067.stdout:(93/120): openblas-threads-0.3.12-1.el8.x86_64. 37 MB/s | 4.7 MB 00:00 2023-10-07T23:04:39.413 INFO:teuthology.orchestra.run.smithi067.stdout:(94/120): python3-pytz-2017.2-9.el8.noarch.rpm 396 kB/s | 54 kB 00:00 2023-10-07T23:04:39.450 INFO:teuthology.orchestra.run.smithi067.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 532 kB/s | 19 kB 00:00 2023-10-07T23:04:39.511 INFO:teuthology.orchestra.run.smithi067.stdout:(96/120): openblas-0.3.12-1.el8.x86_64.rpm 26 MB/s | 4.6 MB 00:00 2023-10-07T23:04:39.524 INFO:teuthology.orchestra.run.smithi067.stdout:(97/120): automake-1.16.1-7.el8.noarch.rpm 6.2 MB/s | 713 kB 00:00 2023-10-07T23:04:39.581 INFO:teuthology.orchestra.run.smithi067.stdout:(98/120): python3-jwt-1.6.1-2.el8.noarch.rpm 322 kB/s | 43 kB 00:00 2023-10-07T23:04:39.638 INFO:teuthology.orchestra.run.smithi067.stdout:(99/120): python3-oauthlib-2.1.0-1.el8.noarch.r 1.2 MB/s | 155 kB 00:00 2023-10-07T23:04:39.646 INFO:teuthology.orchestra.run.smithi067.stdout:(100/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-07T23:04:39.724 INFO:teuthology.orchestra.run.smithi067.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 334 kB/s | 48 kB 00:00 2023-10-07T23:04:39.739 INFO:teuthology.orchestra.run.smithi067.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 457 kB/s | 43 kB 00:00 2023-10-07T23:04:39.809 INFO:teuthology.orchestra.run.smithi067.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2023-10-07T23:04:39.855 INFO:teuthology.orchestra.run.smithi067.stdout:(104/120): openssl-devel-1.1.1g-15.el8_3.x86_64 18 MB/s | 2.3 MB 00:00 2023-10-07T23:04:39.876 INFO:teuthology.orchestra.run.smithi067.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 9.0 MB/s | 605 kB 00:00 2023-10-07T23:04:39.884 INFO:teuthology.orchestra.run.smithi067.stdout:(106/120): libverto-devel-0.3.0-5.el8.x86_64.rp 124 kB/s | 18 kB 00:00 2023-10-07T23:04:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:(107/120): fuse-2.9.7-12.el8.x86_64.rpm 1.5 MB/s | 83 kB 00:00 2023-10-07T23:04:39.923 INFO:teuthology.orchestra.run.smithi067.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 4.3 MB/s | 169 kB 00:00 2023-10-07T23:04:39.928 INFO:teuthology.orchestra.run.smithi067.stdout:(109/120): elfutils-libelf-devel-0.182-3.el8.x8 1.1 MB/s | 59 kB 00:00 2023-10-07T23:04:39.953 INFO:teuthology.orchestra.run.smithi067.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 2.8 MB/s | 86 kB 00:00 2023-10-07T23:04:39.957 INFO:teuthology.orchestra.run.smithi067.stdout:(111/120): libcom_err-devel-1.45.6-1.el8.x86_64 806 kB/s | 38 kB 00:00 2023-10-07T23:04:39.996 INFO:teuthology.orchestra.run.smithi067.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 855 kB/s | 58 kB 00:00 2023-10-07T23:04:40.012 INFO:teuthology.orchestra.run.smithi067.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 9.9 MB/s | 559 kB 00:00 2023-10-07T23:04:40.035 INFO:teuthology.orchestra.run.smithi067.stdout:(114/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 3.0 MB/s | 70 kB 00:00 2023-10-07T23:04:40.045 INFO:teuthology.orchestra.run.smithi067.stdout:(115/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 509 kB/s | 47 kB 00:00 2023-10-07T23:04:40.086 INFO:teuthology.orchestra.run.smithi067.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-10-07T23:04:40.088 INFO:teuthology.orchestra.run.smithi067.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 2.2 MB/s | 97 kB 00:00 2023-10-07T23:04:40.100 INFO:teuthology.orchestra.run.smithi067.stdout:(118/120): libgfortran-8.4.1-1.el8.x86_64.rpm 6.0 MB/s | 639 kB 00:00 2023-10-07T23:04:40.445 INFO:teuthology.orchestra.run.smithi086.stdout:(10/120): ceph-common-17.2.6-1320.g08cac018.el8 1.7 MB/s | 23 MB 00:14 2023-10-07T23:04:40.531 INFO:teuthology.orchestra.run.smithi067.stdout:(119/120): librados2-17.2.6-1320.g08cac018.el8. 7.8 MB/s | 3.5 MB 00:00 2023-10-07T23:04:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 2.2 MB/s | 255 kB 00:00 2023-10-07T23:04:40.608 INFO:teuthology.orchestra.run.smithi086.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 488 kB/s | 25 kB 00:00 2023-10-07T23:04:40.816 INFO:teuthology.orchestra.run.smithi067.stdout:(120/120): librbd1-17.2.6-1320.g08cac018.el8.x8 4.9 MB/s | 3.6 MB 00:00 2023-10-07T23:04:40.819 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:04:40.819 INFO:teuthology.orchestra.run.smithi067.stdout:Total 16 MB/s | 202 MB 00:12 2023-10-07T23:04:40.896 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-10-07T23:04:41.032 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-10-07T23:04:41.032 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-10-07T23:04:41.139 INFO:teuthology.orchestra.run.smithi086.stdout:(13/120): ceph-test-17.2.6-1320.g08cac018.el8.x 14 MB/s | 52 MB 00:03 2023-10-07T23:04:41.256 INFO:teuthology.orchestra.run.smithi086.stdout:(14/120): ceph-radosgw-17.2.6-1320.g08cac018.el 3.0 MB/s | 17 MB 00:05 2023-10-07T23:04:41.258 INFO:teuthology.orchestra.run.smithi086.stdout:(15/120): libcephsqlite-17.2.6-1320.g08cac018.e 1.4 MB/s | 174 kB 00:00 2023-10-07T23:04:41.262 INFO:teuthology.orchestra.run.smithi086.stdout:(16/120): libcephfs2-17.2.6-1320.g08cac018.el8. 1.2 MB/s | 797 kB 00:00 2023-10-07T23:04:41.301 INFO:teuthology.orchestra.run.smithi086.stdout:(17/120): libradosstriper1-17.2.6-1320.g08cac01 12 MB/s | 507 kB 00:00 2023-10-07T23:04:41.323 INFO:teuthology.orchestra.run.smithi086.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 2.1 MB/s | 45 kB 00:00 2023-10-07T23:04:41.328 INFO:teuthology.orchestra.run.smithi086.stdout:(19/120): librados-devel-17.2.6-1320.g08cac018. 1.8 MB/s | 130 kB 00:00 2023-10-07T23:04:41.344 INFO:teuthology.orchestra.run.smithi086.stdout:(20/120): python3-ceph-common-17.2.6-1320.g08ca 5.3 MB/s | 115 kB 00:00 2023-10-07T23:04:41.369 INFO:teuthology.orchestra.run.smithi086.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 15 MB/s | 371 kB 00:00 2023-10-07T23:04:41.371 INFO:teuthology.orchestra.run.smithi086.stdout:(22/120): python3-cephfs-17.2.6-1320.g08cac018. 4.4 MB/s | 190 kB 00:00 2023-10-07T23:04:41.392 INFO:teuthology.orchestra.run.smithi086.stdout:(23/120): python3-rbd-17.2.6-1320.g08cac018.el8 15 MB/s | 358 kB 00:00 2023-10-07T23:04:41.415 INFO:teuthology.orchestra.run.smithi086.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 4.2 MB/s | 90 kB 00:00 2023-10-07T23:04:41.416 INFO:teuthology.orchestra.run.smithi086.stdout:(25/120): python3-rgw-17.2.6-1320.g08cac018.el8 2.3 MB/s | 107 kB 00:00 2023-10-07T23:04:41.465 INFO:teuthology.orchestra.run.smithi086.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 3.5 MB/s | 173 kB 00:00 2023-10-07T23:04:41.517 INFO:teuthology.orchestra.run.smithi086.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 460 kB/s | 23 kB 00:00 2023-10-07T23:04:41.561 INFO:teuthology.orchestra.run.smithi086.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 2.6 MB/s | 121 kB 00:00 2023-10-07T23:04:41.597 INFO:teuthology.orchestra.run.smithi086.stdout:(29/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 19 MB/s | 3.5 MB 00:00 2023-10-07T23:04:41.928 INFO:teuthology.orchestra.run.smithi086.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 4.6 MB/s | 1.7 MB 00:00 2023-10-07T23:04:41.991 INFO:teuthology.orchestra.run.smithi086.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 3.8 MB/s | 238 kB 00:00 2023-10-07T23:04:42.042 INFO:teuthology.orchestra.run.smithi086.stdout:(32/120): ceph-mgr-diskprediction-local-17.2.6- 17 MB/s | 7.4 MB 00:00 2023-10-07T23:04:42.043 INFO:teuthology.orchestra.run.smithi086.stdout:(33/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 912 kB/s | 47 kB 00:00 2023-10-07T23:04:42.065 INFO:teuthology.orchestra.run.smithi086.stdout:(34/120): ceph-prometheus-alerts-17.2.6-1320.g0 658 kB/s | 14 kB 00:00 2023-10-07T23:04:42.087 INFO:teuthology.orchestra.run.smithi086.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 3.6 MB/s | 80 kB 00:00 2023-10-07T23:04:42.089 INFO:teuthology.orchestra.run.smithi086.stdout:(36/120): ceph-volume-17.2.6-1320.g08cac018.el8 5.7 MB/s | 267 kB 00:00 2023-10-07T23:04:42.108 INFO:teuthology.orchestra.run.smithi086.stdout:(37/120): cephfs-top-17.2.6-1320.g08cac018.el8. 865 kB/s | 18 kB 00:00 2023-10-07T23:04:42.167 INFO:teuthology.orchestra.run.smithi086.stdout:(38/120): librgw2-17.2.6-1320.g08cac018.el8.x86 6.1 MB/s | 5.6 MB 00:00 2023-10-07T23:04:42.219 INFO:teuthology.orchestra.run.smithi086.stdout:(39/120): liboath-2.6.2-3.el8.x86_64.rpm 529 kB/s | 59 kB 00:00 2023-10-07T23:04:42.269 INFO:teuthology.orchestra.run.smithi086.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.7 MB/s | 306 kB 00:00 2023-10-07T23:04:42.276 INFO:teuthology.orchestra.run.smithi086.stdout:(41/120): libunwind-1.3.1-3.el8.x86_64.rpm 686 kB/s | 75 kB 00:00 2023-10-07T23:04:42.286 INFO:teuthology.orchestra.run.smithi086.stdout:(42/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 2.5 MB/s | 44 kB 00:00 2023-10-07T23:04:42.304 INFO:teuthology.orchestra.run.smithi086.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.9 MB/s | 33 kB 00:00 2023-10-07T23:04:42.306 INFO:teuthology.orchestra.run.smithi086.stdout:(44/120): python3-asyncssh-2.7.0-2.el8.noarch.r 5.3 MB/s | 476 kB 00:00 2023-10-07T23:04:42.312 INFO:teuthology.orchestra.run.smithi086.stdout:(45/120): python3-beautifulsoup4-4.6.3-2.el8.1. 5.1 MB/s | 185 kB 00:00 2023-10-07T23:04:42.322 INFO:teuthology.orchestra.run.smithi086.stdout:(46/120): python3-certifi-2018.10.15-7.el8.noar 952 kB/s | 16 kB 00:00 2023-10-07T23:04:42.340 INFO:teuthology.orchestra.run.smithi086.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 4.5 MB/s | 81 kB 00:00 2023-10-07T23:04:42.341 INFO:teuthology.orchestra.run.smithi086.stdout:(48/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.8 MB/s | 173 kB 00:00 2023-10-07T23:04:42.357 INFO:teuthology.orchestra.run.smithi086.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 653 kB/s | 11 kB 00:00 2023-10-07T23:04:42.359 INFO:teuthology.orchestra.run.smithi086.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 1.0 MB/s | 18 kB 00:00 2023-10-07T23:04:42.377 INFO:teuthology.orchestra.run.smithi086.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 2.7 MB/s | 49 kB 00:00 2023-10-07T23:04:42.381 INFO:teuthology.orchestra.run.smithi086.stdout:(52/120): python3-cherrypy-18.4.0-1.el8.noarch. 5.4 MB/s | 384 kB 00:00 2023-10-07T23:04:42.395 INFO:teuthology.orchestra.run.smithi086.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 3.2 MB/s | 59 kB 00:00 2023-10-07T23:04:42.399 INFO:teuthology.orchestra.run.smithi086.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 3.3 MB/s | 60 kB 00:00 2023-10-07T23:04:42.417 INFO:teuthology.orchestra.run.smithi086.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 890 kB/s | 16 kB 00:00 2023-10-07T23:04:42.435 INFO:teuthology.orchestra.run.smithi086.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.9 MB/s | 34 kB 00:00 2023-10-07T23:04:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:(57/120): python3-pecan-1.3.2-9.el8.noarch.rpm 5.3 MB/s | 283 kB 00:00 2023-10-07T23:04:42.466 INFO:teuthology.orchestra.run.smithi086.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 3.5 MB/s | 64 kB 00:00 2023-10-07T23:04:42.470 INFO:teuthology.orchestra.run.smithi086.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 5.4 MB/s | 196 kB 00:00 2023-10-07T23:04:42.484 INFO:teuthology.orchestra.run.smithi086.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 1.0 MB/s | 19 kB 00:00 2023-10-07T23:04:42.488 INFO:teuthology.orchestra.run.smithi086.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 1.3 MB/s | 24 kB 00:00 2023-10-07T23:04:42.502 INFO:teuthology.orchestra.run.smithi086.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 1.6 MB/s | 29 kB 00:00 2023-10-07T23:04:42.506 INFO:teuthology.orchestra.run.smithi086.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.5 MB/s | 27 kB 00:00 2023-10-07T23:04:42.546 INFO:teuthology.orchestra.run.smithi086.stdout:(64/120): python3-kubernetes-11.0.0-6.el8.noarc 6.2 MB/s | 1.2 MB 00:00 2023-10-07T23:04:42.553 INFO:teuthology.orchestra.run.smithi086.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 4.8 MB/s | 255 kB 00:00 2023-10-07T23:04:42.559 INFO:teuthology.orchestra.run.smithi086.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 4.7 MB/s | 251 kB 00:00 2023-10-07T23:04:42.563 INFO:teuthology.orchestra.run.smithi086.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 3.4 MB/s | 61 kB 00:00 2023-10-07T23:04:42.572 INFO:teuthology.orchestra.run.smithi086.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 4.6 MB/s | 85 kB 00:00 2023-10-07T23:04:42.576 INFO:teuthology.orchestra.run.smithi086.stdout:(69/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.3 MB/s | 23 kB 00:00 2023-10-07T23:04:42.577 INFO:teuthology.orchestra.run.smithi086.stdout:(70/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 14 MB/s | 70 kB 00:00 2023-10-07T23:04:42.644 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-10-07T23:04:42.649 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-10-07T23:04:42.705 INFO:teuthology.orchestra.run.smithi086.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 345 kB/s | 44 kB 00:00 2023-10-07T23:04:42.780 INFO:teuthology.orchestra.run.smithi086.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 351 kB/s | 27 kB 00:00 2023-10-07T23:04:42.849 INFO:teuthology.orchestra.run.smithi086.stdout:(73/120): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-10-07T23:04:42.875 INFO:teuthology.orchestra.run.smithi086.stdout:(74/120): jq-1.5-12.el8.x86_64.rpm 1.7 MB/s | 161 kB 00:00 2023-10-07T23:04:43.027 INFO:teuthology.orchestra.run.smithi086.stdout:(75/120): python3-scipy-1.0.0-20.module+el8.1.0 32 MB/s | 15 MB 00:00 2023-10-07T23:04:43.035 INFO:teuthology.orchestra.run.smithi086.stdout:(76/120): python3-werkzeug-0.12.2-4.el8.noarch. 2.8 MB/s | 457 kB 00:00 2023-10-07T23:04:43.039 INFO:teuthology.orchestra.run.smithi086.stdout:(77/120): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-10-07T23:04:43.105 INFO:teuthology.orchestra.run.smithi086.stdout:(78/120): python3-markupsafe-0.23-19.el8.x86_64 558 kB/s | 39 kB 00:00 2023-10-07T23:04:43.175 INFO:teuthology.orchestra.run.smithi086.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.6 MB/s | 259 kB 00:00 2023-10-07T23:04:43.186 INFO:teuthology.orchestra.run.smithi086.stdout:(80/120): bison-3.0.4-10.el8.x86_64.rpm 4.6 MB/s | 688 kB 00:00 2023-10-07T23:04:43.226 INFO:teuthology.orchestra.run.smithi086.stdout:(81/120): python3-babel-2.5.1-5.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-10-07T23:04:43.261 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-10-07T23:04:43.294 INFO:teuthology.orchestra.run.smithi086.stdout:(82/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 1.9 MB/s | 225 kB 00:00 2023-10-07T23:04:43.297 INFO:teuthology.orchestra.run.smithi086.stdout:(83/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-10-07T23:04:43.347 INFO:teuthology.orchestra.run.smithi086.stdout:(84/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 31 MB/s | 3.7 MB 00:00 2023-10-07T23:04:43.411 INFO:teuthology.orchestra.run.smithi086.stdout:(85/120): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-10-07T23:04:43.415 INFO:teuthology.orchestra.run.smithi086.stdout:(86/120): python3-jinja2-2.10.1-2.el8_0.noarch. 4.4 MB/s | 538 kB 00:00 2023-10-07T23:04:43.461 INFO:teuthology.orchestra.run.smithi086.stdout:(87/120): socat-1.7.3.3-2.el8.x86_64.rpm 2.6 MB/s | 302 kB 00:00 2023-10-07T23:04:43.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-07T23:04:43.513 INFO:teuthology.orchestra.run.smithi086.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-10-07T23:04:43.540 INFO:teuthology.orchestra.run.smithi086.stdout:(89/120): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-10-07T23:04:43.589 INFO:teuthology.orchestra.run.smithi086.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 2.7 MB/s | 353 kB 00:00 2023-10-07T23:04:43.621 INFO:teuthology.orchestra.run.smithi086.stdout:(91/120): python3-jsonpointer-1.10-11.el8.noarc 185 kB/s | 20 kB 00:00 2023-10-07T23:04:43.660 INFO:teuthology.orchestra.run.smithi086.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 5.8 MB/s | 709 kB 00:00 2023-10-07T23:04:43.697 INFO:teuthology.orchestra.run.smithi086.stdout:(93/120): python3-pytz-2017.2-9.el8.noarch.rpm 504 kB/s | 54 kB 00:00 2023-10-07T23:04:43.736 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-07T23:04:43.752 INFO:teuthology.orchestra.run.smithi086.stdout:(94/120): openblas-threads-0.3.12-1.el8.x86_64. 36 MB/s | 4.7 MB 00:00 2023-10-07T23:04:43.759 INFO:teuthology.orchestra.run.smithi086.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2023-10-07T23:04:43.784 INFO:teuthology.orchestra.run.smithi086.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 580 kB/s | 19 kB 00:00 2023-10-07T23:04:43.826 INFO:teuthology.orchestra.run.smithi086.stdout:(97/120): openblas-0.3.12-1.el8.x86_64.rpm 28 MB/s | 4.6 MB 00:00 2023-10-07T23:04:43.828 INFO:teuthology.orchestra.run.smithi086.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 3.4 MB/s | 155 kB 00:00 2023-10-07T23:04:43.833 INFO:teuthology.orchestra.run.smithi086.stdout:(99/120): python3-jwt-1.6.1-2.el8.noarch.rpm 575 kB/s | 43 kB 00:00 2023-10-07T23:04:43.858 INFO:teuthology.orchestra.run.smithi086.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 1.5 MB/s | 48 kB 00:00 2023-10-07T23:04:43.870 INFO:teuthology.orchestra.run.smithi086.stdout:(101/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.9 MB/s | 220 kB 00:00 2023-10-07T23:04:43.889 INFO:teuthology.orchestra.run.smithi086.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 1.4 MB/s | 43 kB 00:00 2023-10-07T23:04:43.906 INFO:teuthology.orchestra.run.smithi086.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.1 MB/s | 229 kB 00:00 2023-10-07T23:04:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-07T23:04:43.925 INFO:teuthology.orchestra.run.smithi086.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 501 kB/s | 18 kB 00:00 2023-10-07T23:04:43.949 INFO:teuthology.orchestra.run.smithi086.stdout:(105/120): openssl-devel-1.1.1g-15.el8_3.x86_64 30 MB/s | 2.3 MB 00:00 2023-10-07T23:04:43.951 INFO:teuthology.orchestra.run.smithi086.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 3.1 MB/s | 83 kB 00:00 2023-10-07T23:04:43.956 INFO:teuthology.orchestra.run.smithi086.stdout:(107/120): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-10-07T23:04:43.977 INFO:teuthology.orchestra.run.smithi086.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 2.1 MB/s | 59 kB 00:00 2023-10-07T23:04:43.998 INFO:teuthology.orchestra.run.smithi086.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 3.6 MB/s | 169 kB 00:00 2023-10-07T23:04:44.004 INFO:teuthology.orchestra.run.smithi086.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 809 kB/s | 38 kB 00:00 2023-10-07T23:04:44.019 INFO:teuthology.orchestra.run.smithi086.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 2.0 MB/s | 86 kB 00:00 2023-10-07T23:04:44.035 INFO:teuthology.orchestra.run.smithi086.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 1.5 MB/s | 58 kB 00:00 2023-10-07T23:04:44.046 INFO:teuthology.orchestra.run.smithi086.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-10-07T23:04:44.057 INFO:teuthology.orchestra.run.smithi086.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 14 MB/s | 559 kB 00:00 2023-10-07T23:04:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-07T23:04:44.090 INFO:teuthology.orchestra.run.smithi086.stdout:(115/120): libgfortran-8.4.1-1.el8.x86_64.rpm 11 MB/s | 639 kB 00:00 2023-10-07T23:04:44.091 INFO:teuthology.orchestra.run.smithi086.stdout:(116/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 1.5 MB/s | 70 kB 00:00 2023-10-07T23:04:44.110 INFO:teuthology.orchestra.run.smithi086.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 4.7 MB/s | 97 kB 00:00 2023-10-07T23:04:44.141 INFO:teuthology.orchestra.run.smithi086.stdout:(118/120): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2023-10-07T23:04:44.280 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-07T23:04:44.661 INFO:teuthology.orchestra.run.smithi086.stdout:(119/120): librbd1-17.2.6-1320.g08cac018.el8.x8 6.5 MB/s | 3.6 MB 00:00 2023-10-07T23:04:44.697 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-07T23:04:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-07T23:04:44.922 INFO:teuthology.orchestra.run.smithi086.stdout:(120/120): librados2-17.2.6-1320.g08cac018.el8. 4.2 MB/s | 3.5 MB 00:00 2023-10-07T23:04:44.928 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:04:44.929 INFO:teuthology.orchestra.run.smithi086.stdout:Total 11 MB/s | 202 MB 00:18 2023-10-07T23:04:44.977 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2023-10-07T23:04:45.090 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2023-10-07T23:04:45.090 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2023-10-07T23:04:45.286 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-07T23:04:45.530 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:04:45.663 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:04:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:04:45.980 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:04:46.366 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-07T23:04:46.524 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2023-10-07T23:04:46.528 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2023-10-07T23:04:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-07T23:04:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-10-07T23:04:47.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:04:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:04:47.593 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-07T23:04:47.784 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-07T23:04:47.856 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-07T23:04:47.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-07T23:04:48.049 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-07T23:04:48.176 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-07T23:04:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:04:48.341 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:04:48.416 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-07T23:04:48.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-07T23:04:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-07T23:04:48.761 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-07T23:04:48.883 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-07T23:04:48.930 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-07T23:04:49.056 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-07T23:04:49.066 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-07T23:04:49.249 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-07T23:04:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-07T23:04:49.786 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-07T23:04:49.790 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:04:49.955 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:04:49.996 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-07T23:04:50.140 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:04:50.270 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:04:50.508 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-07T23:04:50.669 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-07T23:04:50.713 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-07T23:04:51.440 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-07T23:04:51.449 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-07T23:04:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:04:51.741 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-07T23:04:51.744 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2023-10-07T23:04:51.756 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:04:51.894 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-07T23:04:51.909 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-07T23:04:52.031 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-07T23:04:52.109 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-07T23:04:52.138 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-07T23:04:52.200 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-07T23:04:52.306 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-07T23:04:52.440 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:04:52.449 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-07T23:04:52.570 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-07T23:04:52.573 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:04:52.656 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-07T23:04:52.782 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-07T23:04:52.894 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-07T23:04:52.963 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-07T23:04:53.058 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-07T23:04:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-07T23:04:53.160 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-07T23:04:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-07T23:04:53.298 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-07T23:04:53.311 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-07T23:04:53.446 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-07T23:04:53.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-07T23:04:53.606 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-07T23:04:53.708 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-07T23:04:53.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-07T23:04:53.939 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:04:54.065 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:04:54.239 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:04:54.250 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-07T23:04:54.335 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:04:54.532 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-07T23:04:54.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-07T23:04:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-07T23:04:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-07T23:04:55.151 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-07T23:04:55.340 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-07T23:04:55.363 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-07T23:04:55.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-07T23:04:55.556 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:04:55.618 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-07T23:04:55.676 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:04:55.918 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-07T23:04:55.920 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-07T23:04:56.076 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-07T23:04:56.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-07T23:04:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-07T23:04:56.272 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:04:56.367 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:04:56.517 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-07T23:04:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-07T23:04:56.637 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-07T23:04:56.760 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-07T23:04:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-07T23:04:56.923 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-07T23:04:56.956 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-07T23:04:57.085 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-07T23:04:57.272 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-07T23:04:57.757 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-07T23:04:57.818 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-07T23:04:57.997 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-07T23:04:58.053 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-07T23:04:58.252 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-07T23:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-07T23:04:58.526 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-07T23:04:58.600 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-07T23:04:58.631 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-07T23:04:58.754 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-07T23:04:58.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-07T23:04:58.947 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-07T23:04:59.014 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-07T23:04:59.037 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-07T23:04:59.263 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-07T23:04:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-07T23:04:59.644 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-07T23:04:59.712 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-07T23:04:59.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-07T23:04:59.974 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-07T23:04:59.975 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-07T23:05:00.121 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-07T23:05:00.188 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-07T23:05:00.239 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-07T23:05:00.322 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-07T23:05:00.390 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-07T23:05:00.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-07T23:05:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-07T23:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-07T23:05:00.686 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-07T23:05:00.823 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-07T23:05:00.903 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-07T23:05:00.932 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-07T23:05:01.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-07T23:05:01.175 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-07T23:05:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-07T23:05:01.337 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-07T23:05:01.443 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-07T23:05:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-07T23:05:01.499 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-07T23:05:01.714 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-07T23:05:01.723 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-07T23:05:01.745 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-07T23:05:02.086 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-07T23:05:02.276 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-07T23:05:02.468 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-07T23:05:02.680 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-07T23:05:02.769 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-07T23:05:02.848 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-07T23:05:03.118 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-07T23:05:03.409 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-07T23:05:03.431 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-07T23:05:03.493 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-07T23:05:03.654 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-07T23:05:03.694 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-07T23:05:03.874 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-07T23:05:03.922 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-07T23:05:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-07T23:05:04.121 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-07T23:05:04.245 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-07T23:05:04.284 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-07T23:05:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-07T23:05:04.636 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-07T23:05:04.712 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-07T23:05:04.875 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-07T23:05:04.929 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-07T23:05:05.075 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-07T23:05:05.091 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-07T23:05:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-07T23:05:05.306 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-07T23:05:05.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-07T23:05:05.530 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-07T23:05:05.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-07T23:05:05.700 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-07T23:05:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-07T23:05:05.876 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-07T23:05:06.031 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-07T23:05:06.070 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-07T23:05:06.179 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-07T23:05:06.201 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-07T23:05:06.254 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-07T23:05:06.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-07T23:05:06.501 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-07T23:05:06.526 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-07T23:05:06.639 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-07T23:05:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-07T23:05:06.769 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-07T23:05:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-07T23:05:06.900 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-07T23:05:06.939 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-07T23:05:07.023 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-07T23:05:07.165 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-07T23:05:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-07T23:05:07.326 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-07T23:05:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-07T23:05:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-07T23:05:07.537 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-07T23:05:07.655 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-07T23:05:07.704 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-07T23:05:07.746 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-07T23:05:07.899 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-07T23:05:07.909 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-07T23:05:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-07T23:05:08.098 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-07T23:05:08.257 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-07T23:05:08.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-07T23:05:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-07T23:05:08.604 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-07T23:05:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-07T23:05:08.800 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-07T23:05:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-07T23:05:09.032 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-07T23:05:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-07T23:05:09.108 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-07T23:05:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-07T23:05:09.367 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-07T23:05:09.405 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-07T23:05:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-07T23:05:09.570 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-07T23:05:09.614 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-07T23:05:09.651 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-07T23:05:09.826 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-07T23:05:09.920 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-07T23:05:10.011 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-07T23:05:10.062 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-07T23:05:10.209 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-07T23:05:10.220 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-07T23:05:10.245 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-07T23:05:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-07T23:05:10.453 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-07T23:05:10.566 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-07T23:05:10.642 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-07T23:05:10.738 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-07T23:05:10.832 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-07T23:05:10.858 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-07T23:05:10.993 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-07T23:05:11.012 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-07T23:05:11.018 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-07T23:05:11.090 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-07T23:05:11.167 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-07T23:05:11.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-07T23:05:11.279 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-07T23:05:11.312 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-07T23:05:11.484 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-07T23:05:11.496 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-07T23:05:11.622 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-07T23:05:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-07T23:05:11.777 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-07T23:05:12.030 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-07T23:05:12.055 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-07T23:05:12.262 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-07T23:05:12.262 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-07T23:05:12.453 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-07T23:05:12.535 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-07T23:05:12.626 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-07T23:05:12.681 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-07T23:05:12.827 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-07T23:05:12.978 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-07T23:05:13.027 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-07T23:05:13.206 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-07T23:05:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-07T23:05:13.456 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-07T23:05:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-07T23:05:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-07T23:05:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-07T23:05:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-07T23:05:14.032 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-07T23:05:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-07T23:05:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-07T23:05:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-07T23:05:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-07T23:05:14.542 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-07T23:05:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-07T23:05:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-07T23:05:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-07T23:05:15.477 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-07T23:05:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-07T23:05:16.296 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-07T23:05:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-07T23:05:16.655 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-07T23:05:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-07T23:05:16.942 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-07T23:05:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-07T23:05:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-07T23:05:17.452 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-07T23:05:17.485 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-07T23:05:17.589 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-07T23:05:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-07T23:05:17.709 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-07T23:05:17.759 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-07T23:05:17.901 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-07T23:05:18.050 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-07T23:05:18.086 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-07T23:05:18.288 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-07T23:05:18.307 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-07T23:05:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-07T23:05:18.728 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-07T23:05:23.917 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:24.128 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:24.706 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:25.468 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-07T23:05:25.583 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-07T23:05:25.744 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-07T23:05:25.912 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:26.129 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:26.831 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-07T23:05:27.096 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-07T23:05:27.315 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-07T23:05:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-07T23:05:35.655 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-07T23:05:35.655 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2023-10-07T23:05:35.655 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2023-10-07T23:05:35.655 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-10-07T23:05:35.656 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:05:36.031 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-07T23:05:37.510 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-07T23:05:37.699 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-07T23:05:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-07T23:05:38.093 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-07T23:05:38.481 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-07T23:05:38.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-07T23:05:38.664 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-07T23:05:41.606 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-07T23:05:41.858 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-07T23:05:41.969 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-07T23:05:42.169 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-07T23:05:42.338 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-07T23:05:42.555 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-07T23:05:42.806 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /run 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2023-10-07T23:05:43.404 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2023-10-07T23:05:43.405 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2023-10-07T23:05:43.405 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:05:43.742 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-07T23:05:43.769 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-07T23:05:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-07T23:05:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-07T23:05:44.421 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-07T23:05:44.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-07T23:05:44.652 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-07T23:05:44.837 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-07T23:05:44.848 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-07T23:05:45.228 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-07T23:05:45.382 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-07T23:05:45.603 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-07T23:05:45.768 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-07T23:05:46.155 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-07T23:05:46.325 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-07T23:05:46.341 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-07T23:05:46.448 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-07T23:05:46.656 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-07T23:05:46.749 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-07T23:05:46.938 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-07T23:05:46.980 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-07T23:05:47.212 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-07T23:05:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-07T23:05:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2023-10-07T23:05:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2023-10-07T23:05:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2023-10-07T23:05:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /run 2023-10-07T23:05:47.467 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2023-10-07T23:05:47.467 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2023-10-07T23:05:47.467 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2023-10-07T23:05:47.467 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2023-10-07T23:05:47.467 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2023-10-07T23:05:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:05:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-07T23:05:47.729 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-07T23:05:47.875 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-07T23:05:49.178 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-07T23:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-07T23:05:49.547 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-07T23:05:49.605 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-07T23:05:49.753 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-07T23:05:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-07T23:05:49.924 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-07T23:05:50.072 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-07T23:05:50.138 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-07T23:05:50.277 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-07T23:05:50.356 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-07T23:05:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-07T23:05:50.917 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-07T23:05:50.959 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-07T23:05:51.316 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-07T23:05:51.532 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-07T23:05:51.630 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-07T23:05:52.010 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-07T23:05:52.223 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-07T23:05:52.259 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-07T23:05:52.465 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-07T23:05:52.476 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-07T23:05:52.647 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-07T23:05:53.008 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-07T23:05:53.144 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-07T23:05:53.835 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-07T23:05:54.081 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-07T23:05:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-07T23:05:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-07T23:05:54.360 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-07T23:05:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-07T23:05:54.541 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-07T23:05:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-07T23:05:54.583 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-07T23:05:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-07T23:05:54.801 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-07T23:05:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-07T23:05:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-07T23:05:55.146 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-07T23:05:55.210 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-07T23:05:55.315 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-07T23:05:55.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-07T23:05:55.596 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-07T23:05:55.610 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-07T23:05:55.787 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-07T23:05:55.810 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-07T23:05:55.867 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-07T23:05:56.025 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-07T23:05:56.036 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-07T23:05:56.044 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-07T23:05:56.045 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-07T23:05:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-07T23:05:56.271 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-07T23:05:56.272 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:05:56.432 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-07T23:05:56.712 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-07T23:05:56.925 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-07T23:05:57.145 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:05:57.145 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-07T23:05:57.145 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-07T23:05:57.146 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-07T23:05:57.147 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-07T23:05:57.148 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-07T23:05:57.149 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-07T23:05:57.150 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-07T23:05:57.151 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-07T23:05:57.151 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-07T23:05:57.151 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-07T23:05:57.151 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-07T23:05:57.151 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-07T23:05:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-07T23:05:57.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-07T23:05:57.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-07T23:05:57.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-07T23:05:57.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-07T23:05:57.157 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-07T23:05:57.157 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-07T23:05:57.157 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-07T23:05:57.157 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-07T23:05:57.157 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-07T23:05:58.111 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-07T23:05:58.291 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-07T23:05:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-07T23:05:58.836 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-07T23:05:59.026 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-07T23:05:59.096 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-07T23:05:59.240 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-07T23:05:59.269 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-07T23:05:59.512 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:05:59.512 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:43 ago on Sat 07 Oct 2023 11:04:16 PM UTC. 2023-10-07T23:06:00.215 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-07T23:06:00.549 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-07T23:06:00.697 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-07T23:06:01.147 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-07T23:06:01.375 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-07T23:06:01.384 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-07T23:06:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-07T23:06:01.674 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-07T23:06:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-07T23:06:01.765 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-07T23:06:01.932 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-07T23:06:01.967 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-07T23:06:02.126 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-07T23:06:02.283 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-07T23:06:02.367 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-07T23:06:02.427 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-07T23:06:02.549 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-07T23:06:02.572 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-07T23:06:02.772 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-07T23:06:02.907 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-07T23:06:03.174 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-07T23:06:03.380 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-07T23:06:03.446 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-07T23:06:03.622 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-07T23:06:03.632 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-07T23:06:03.632 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-07T23:06:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-07T23:06:03.850 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:06:04.711 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:06:04.712 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-07T23:06:04.713 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-07T23:06:04.714 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-07T23:06:04.715 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-07T23:06:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-07T23:06:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-07T23:06:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-07T23:06:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-07T23:06:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-07T23:06:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-07T23:06:04.722 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-07T23:06:04.723 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-07T23:06:04.724 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-07T23:06:04.725 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-07T23:06:04.725 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-07T23:06:05.797 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.1-7.el8.noarch 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.0.4-10.el8.x86_64 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.921 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.923 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.1-9.el8.x86_64 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-07T23:06:05.924 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.5-12.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-07T23:06:05.925 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-07T23:06:05.926 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-07T23:06:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-07T23:06:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-07T23:06:05.929 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-07T23:06:05.930 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-07T23:06:05.931 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-07T23:06:05.932 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-07T23:06:05.933 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:06:05.934 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-10-07T23:06:06.121 DEBUG:teuthology.parallel:result is None 2023-10-07T23:06:07.086 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:06:07.086 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:51 ago on Sat 07 Oct 2023 11:04:16 PM UTC. 2023-10-07T23:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-07T23:06:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-07T23:06:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-07T23:06:10.059 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-07T23:06:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-07T23:06:10.739 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-07T23:06:10.902 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-07T23:06:11.087 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-07T23:06:11.229 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-07T23:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-07T23:06:11.579 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2023-10-07T23:06:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.1-7.el8.noarch 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.0.4-10.el8.x86_64 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.702 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.703 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-07T23:06:11.704 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.1-9.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.5-12.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-07T23:06:11.705 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-07T23:06:11.706 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-07T23:06:11.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-07T23:06:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-07T23:06:11.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-2.6-1.el8.noarch 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-07T23:06:11.710 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-07T23:06:11.711 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-07T23:06:11.712 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-07T23:06:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-07T23:06:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-07T23:06:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-07T23:06:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2023-10-07T23:06:11.766 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-07T23:06:11.893 DEBUG:teuthology.parallel:result is None 2023-10-07T23:06:12.004 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-07T23:06:12.205 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-07T23:06:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-07T23:06:12.464 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-07T23:06:12.489 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-07T23:06:12.489 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-07T23:06:12.771 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-07T23:06:12.771 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:06:13.177 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:06:13.178 INFO:teuthology.orchestra.run.smithi067.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-07T23:06:13.178 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:06:14.019 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-07T23:06:14.020 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-07T23:06:14.021 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-07T23:06:14.021 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-07T23:06:14.021 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-07T23:06:14.021 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-07T23:06:14.021 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-07T23:06:14.021 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-07T23:06:14.022 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-07T23:06:14.022 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-07T23:06:14.022 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-07T23:06:14.022 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-07T23:06:14.022 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-07T23:06:14.023 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-07T23:06:14.023 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-07T23:06:14.023 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-07T23:06:14.023 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-07T23:06:14.023 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-07T23:06:14.023 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-07T23:06:14.024 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-07T23:06:14.024 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-07T23:06:14.024 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-07T23:06:14.024 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-07T23:06:14.024 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-07T23:06:14.024 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-07T23:06:14.025 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-07T23:06:14.025 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-07T23:06:14.025 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-07T23:06:14.025 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-07T23:06:14.025 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-07T23:06:14.025 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-07T23:06:14.026 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-07T23:06:14.026 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-07T23:06:14.026 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-07T23:06:14.026 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-07T23:06:14.026 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-07T23:06:14.026 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-07T23:06:14.027 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-07T23:06:14.027 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-07T23:06:14.027 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-07T23:06:14.029 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-07T23:06:14.029 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-07T23:06:14.029 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-07T23:06:14.029 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-07T23:06:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-07T23:06:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-07T23:06:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-07T23:06:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-07T23:06:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-07T23:06:14.030 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-07T23:06:14.031 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-07T23:06:14.031 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-07T23:06:14.031 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-07T23:06:14.031 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-07T23:06:14.031 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-07T23:06:14.031 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-07T23:06:14.032 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-07T23:06:14.032 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-07T23:06:14.032 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-07T23:06:14.032 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-07T23:06:14.032 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-07T23:06:14.033 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-07T23:06:14.033 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-07T23:06:14.033 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-07T23:06:14.033 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-07T23:06:14.033 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-07T23:06:14.033 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-07T23:06:14.034 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-07T23:06:14.034 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-07T23:06:14.034 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-07T23:06:14.034 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-07T23:06:14.034 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-07T23:06:14.034 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-07T23:06:14.035 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-07T23:06:14.035 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-07T23:06:14.035 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-07T23:06:14.035 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-07T23:06:14.035 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-07T23:06:14.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-07T23:06:14.037 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-07T23:06:14.037 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-07T23:06:14.037 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-07T23:06:14.037 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-07T23:06:14.037 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-07T23:06:14.037 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-07T23:06:14.038 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-07T23:06:14.038 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-07T23:06:14.038 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-07T23:06:14.038 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-07T23:06:14.038 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-07T23:06:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-07T23:06:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-07T23:06:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-07T23:06:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-07T23:06:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-07T23:06:14.039 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-07T23:06:14.040 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-07T23:06:14.040 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-07T23:06:14.040 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-07T23:06:14.040 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-07T23:06:14.040 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-07T23:06:14.040 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-07T23:06:14.041 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-07T23:06:14.041 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-07T23:06:14.041 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-07T23:06:14.041 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-07T23:06:14.041 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-07T23:06:14.041 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-07T23:06:14.042 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-07T23:06:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-07T23:06:16.663 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:58 ago on Sat 07 Oct 2023 11:04:18 PM UTC. 2023-10-07T23:06:20.752 INFO:teuthology.orchestra.run.smithi067.stdout:Installed products updated. 2023-10-07T23:06:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:06:20.890 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2023-10-07T23:06:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.1-7.el8.noarch 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.0.4-10.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.1-9.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.5-12.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-07T23:06:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-07T23:06:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-07T23:06:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-07T23:06:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-07T23:06:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-07T23:06:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-07T23:06:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-2.6-1.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-07T23:06:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-07T23:06:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-07T23:06:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-07T23:06:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:06:20.905 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-10-07T23:06:21.490 DEBUG:teuthology.parallel:result is None 2023-10-07T23:06:21.490 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:06:21.706 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-07T23:06:21.755 INFO:teuthology.orchestra.run.smithi067.stdout:17.2.6-1320.g08cac018.el8 2023-10-07T23:06:21.755 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-07T23:06:21.756 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-07T23:06:21.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:06:21.969 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-07T23:06:21.994 INFO:teuthology.orchestra.run.smithi086.stdout:17.2.6-1320.g08cac018.el8 2023-10-07T23:06:21.994 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-07T23:06:21.994 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-07T23:06:21.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:06:22.134 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-07T23:06:22.159 INFO:teuthology.orchestra.run.smithi123.stdout:17.2.6-1320.g08cac018.el8 2023-10-07T23:06:22.159 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-07T23:06:22.159 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-07T23:06:22.159 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-07T23:06:22.159 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:06:22.160 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T23:06:22.247 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:06:22.248 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T23:06:22.291 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:06:22.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T23:06:22.334 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-07T23:06:22.335 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:06:22.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T23:06:22.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T23:06:22.500 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:06:22.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T23:06:22.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T23:06:22.598 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:06:22.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T23:06:22.626 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T23:06:22.693 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-07T23:06:22.693 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:06:22.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T23:06:22.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T23:06:22.856 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:06:22.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T23:06:22.884 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T23:06:22.950 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:06:22.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T23:06:22.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T23:06:23.043 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-07T23:06:23.043 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:06:23.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T23:06:23.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T23:06:23.206 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:06:23.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T23:06:23.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T23:06:23.302 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:06:23.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T23:06:23.331 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T23:06:23.400 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-07T23:06:23.478 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-07T23:06:23.480 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:06:23.481 INFO:tasks.cephadm:Cluster fsid is 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:06:23.481 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-07T23:06:23.481 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.86', 'mon.c': '172.21.15.123'} 2023-10-07T23:06:23.481 INFO:tasks.cephadm:First mon is mon.a on smithi067 2023-10-07T23:06:23.481 INFO:tasks.cephadm:First mgr is x 2023-10-07T23:06:23.481 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-07T23:06:23.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2023-10-07T23:06:23.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2023-10-07T23:06:23.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2023-10-07T23:06:23.602 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph-ci.git ref 08cac0187919fc3a247efdf445f652e2fb30a647)... 2023-10-07T23:06:23.603 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/08cac0187919fc3a247efdf445f652e2fb30a647/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:23.834 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366887 Oct 7 23:06 /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:23.834 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/08cac0187919fc3a247efdf445f652e2fb30a647/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:23.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366887 Oct 7 23:06 /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:23.978 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/08cac0187919fc3a247efdf445f652e2fb30a647/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:24.172 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366887 Oct 7 23:06 /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:24.172 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:24.213 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:24.232 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-07T23:06:24.264 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 on all hosts... 2023-10-07T23:06:24.265 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 pull 2023-10-07T23:06:24.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 pull 2023-10-07T23:06:24.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 pull 2023-10-07T23:06:26.720 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647... 2023-10-07T23:06:28.981 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647... 2023-10-07T23:06:29.343 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647... 2023-10-07T23:07:01.799 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:07:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:07:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "5b7bdd6009736d71f57a2d945ac4c228d3efcf438a8f1f55ee05e3a13a875731", 2023-10-07T23:07:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2023-10-07T23:07:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fbbfaf5710669a2fab12abf15fa5c9845aab3690386b0f1ba5057e5eceec1992" 2023-10-07T23:07:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2023-10-07T23:07:01.800 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "5b7bdd6009736d71f57a2d945ac4c228d3efcf438a8f1f55ee05e3a13a875731", 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fbbfaf5710669a2fab12abf15fa5c9845aab3690386b0f1ba5057e5eceec1992" 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2023-10-07T23:07:12.505 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-07T23:07:14.954 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-07T23:07:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:07:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "5b7bdd6009736d71f57a2d945ac4c228d3efcf438a8f1f55ee05e3a13a875731", 2023-10-07T23:07:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2023-10-07T23:07:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fbbfaf5710669a2fab12abf15fa5c9845aab3690386b0f1ba5057e5eceec1992" 2023-10-07T23:07:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2023-10-07T23:07:14.955 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-07T23:07:14.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2023-10-07T23:07:15.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2023-10-07T23:07:15.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2023-10-07T23:07:15.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2023-10-07T23:07:15.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2023-10-07T23:07:15.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2023-10-07T23:07:15.241 INFO:tasks.cephadm:Writing seed config... 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [client] debug client = 20 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-10-07T23:07:15.242 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-07T23:07:15.243 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-07T23:07:15.244 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-07T23:07:15.245 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-07T23:07:15.245 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-07T23:07:15.245 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-07T23:07:15.245 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-10-07T23:07:15.245 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-07T23:07:15.245 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-07T23:07:15.246 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:07:15.246 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-07T23:07:15.283 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 227de690-6566-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-10-07T23:07:15.284 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-07T23:07:15.328 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-07T23:07:15.374 INFO:tasks.cephadm:Bootstrapping... 2023-10-07T23:07:15.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 -v bootstrap --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:07:15.389 INFO:journalctl@ceph.mon.a.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:07:15.435 INFO:journalctl@ceph.mgr.x.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:07:15.647 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-10-07T23:07:15.647 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647', '-v', 'bootstrap', '--fsid', '227de690-6566-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2023-10-07T23:07:15.818 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 3.2.3 2023-10-07T23:07:15.818 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-07T23:07:15.819 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2023-10-07T23:07:15.987 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 3.2.3 2023-10-07T23:07:15.987 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2023-10-07T23:07:15.987 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2023-10-07T23:07:16.002 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-07T23:07:16.002 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-07T23:07:16.015 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-07T23:07:16.015 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2023-10-07T23:07:16.029 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2023-10-07T23:07:16.042 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2023-10-07T23:07:16.042 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2023-10-07T23:07:16.042 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2023-10-07T23:07:16.210 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 3.2.3 2023-10-07T23:07:16.210 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 3.2.3 is present 2023-10-07T23:07:16.210 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2023-10-07T23:07:16.210 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2023-10-07T23:07:16.224 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-07T23:07:16.224 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-07T23:07:16.236 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-07T23:07:16.237 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2023-10-07T23:07:16.251 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2023-10-07T23:07:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2023-10-07T23:07:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2023-10-07T23:07:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2023-10-07T23:07:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 140693247543224 on /run/cephadm/227de690-6566-11ee-8db5-212e2dc638e7.lock 2023-10-07T23:07:16.265 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 140693247543224 acquired on /run/cephadm/227de690-6566-11ee-8db5-212e2dc638e7.lock 2023-10-07T23:07:16.265 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2023-10-07T23:07:16.265 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2023-10-07T23:07:16.265 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2023-10-07T23:07:16.273 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-10-07T23:07:16.273 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2023-10-07T23:07:16.280 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-07T23:07:16.280 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2023-10-07T23:07:16.280 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2023-10-07T23:07:16.286 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-07T23:07:16.286 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-07T23:07:16.286 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-07T23:07:16.287 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-10-07T23:07:16.287 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2023-10-07T23:07:16.287 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-07T23:07:16.287 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2023-10-07T23:07:16.287 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2023-10-07T23:07:16.288 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-07T23:07:16.288 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-07T23:07:16.289 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647... 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5b7bdd6009736d71f57a2d945ac4c228d3efcf438a8f1f55ee05e3a13a875731 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647... 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:8bce0db2f6df548b807b6b3396ec83084ba7197391f11c16ad9841cefbbaa1e8 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:5b7bdd6009736d71f57a2d945ac4c228d3efcf438a8f1f55ee05e3a13a875731 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-07T23:07:16.760 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Storing signatures 2023-10-07T23:07:17.630 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable) 2023-10-07T23:07:17.630 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable) 2023-10-07T23:07:17.630 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2023-10-07T23:07:18.402 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2023-10-07T23:07:18.402 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2023-10-07T23:07:19.171 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQCm5CFly12ULxAAPAwLJolUWwZ98Uzp9TZWgA== 2023-10-07T23:07:20.042 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQCn5CFlweMVIRAAtlWZKmkOGxU4yWrC5AwQ4Q== 2023-10-07T23:07:29.723 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQCo5CFlzOXqGBAAfzVYdHWCKMkMNV6t42eUVQ== 2023-10-07T23:07:29.724 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2023-10-07T23:07:30.497 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = octopus 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:07:30.498 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2023-10-07T23:07:31.268 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2023-10-07T23:07:31.520 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-07T23:07:31.742 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-07T23:07:31.977 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-227de690-6566-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-227de690-6566-11ee-8db5-212e2dc638e7.target. 2023-10-07T23:07:31.977 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-227de690-6566-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-227de690-6566-11ee-8db5-212e2dc638e7.target. 2023-10-07T23:07:32.332 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a 2023-10-07T23:07:32.332 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-07T23:07:32.581 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-227de690-6566-11ee-8db5-212e2dc638e7.target.wants/ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-227de690-6566-11ee-8db5-212e2dc638e7@.service. 2023-10-07T23:07:33.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:32 smithi067 systemd[1]: Starting Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:07:33.476 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-07T23:07:33.476 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2023-10-07T23:07:33.493 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-07T23:07:33.494 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2023-10-07T23:07:33.494 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2023-10-07T23:07:33.494 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2023-10-07T23:07:33.494 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2023-10-07T23:07:33.495 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2023-10-07T23:07:33.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:33 smithi067 bash[81738]: f5d4ba8fd9994311956fc5d27dd98051e710c258c0cadb7f3fdc62a97565b74d 2023-10-07T23:07:33.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:33 smithi067 ceph-mon[81801]: mkfs 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:33.738 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:33 smithi067 ceph-mon[81801]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-07T23:07:33.738 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:33 smithi067 systemd[1]: Started Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:07:34.751 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:34 smithi067 ceph-mon[81801]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-07T23:07:34.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:34 smithi067 ceph-mon[81801]: monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} 2023-10-07T23:07:34.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:34 smithi067 ceph-mon[81801]: fsmap 2023-10-07T23:07:34.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:34 smithi067 ceph-mon[81801]: osdmap e1: 0 total, 0 up, 0 in 2023-10-07T23:07:34.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:34 smithi067 ceph-mon[81801]: mgrmap e1: no daemons active 2023-10-07T23:07:34.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:34 smithi067 ceph-mon[81801]: from='client.? 172.21.15.67:0/3753592973' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-07T23:07:34.769 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2023-10-07T23:07:34.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.638384s) 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-07T23:07:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.986+0000 7fd0c183d700 1 Processor -- start 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.986+0000 7fd0c183d700 1 -- start start 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.987+0000 7fd0c183d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.987+0000 7fd0c183d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc105180 con 0x7fd0bc1047a0 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.987+0000 7fd0baffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0baffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54306/0 (socket says 172.21.15.67:54306) 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0baffd700 1 -- 172.21.15.67:0/2348290940 learned_addr learned my addr 172.21.15.67:0/2348290940 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0baffd700 1 -- 172.21.15.67:0/2348290940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc1052c0 con 0x7fd0bc1047a0 2023-10-07T23:07:34.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0baffd700 1 --2- 172.21.15.67:0/2348290940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc104bb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0098e0 tx=0x7fd0ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2823367d9f116a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0b9ffb700 1 -- 172.21.15.67:0/2348290940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac009e80 con 0x7fd0bc1047a0 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0b9ffb700 1 -- 172.21.15.67:0/2348290940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac00b860 con 0x7fd0bc1047a0 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.988+0000 7fd0b9ffb700 1 -- 172.21.15.67:0/2348290940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac00b9d0 con 0x7fd0bc1047a0 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 -- 172.21.15.67:0/2348290940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 msgr2=0x7fd0bc104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 --2- 172.21.15.67:0/2348290940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc104bb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0098e0 tx=0x7fd0ac009250 comp rx=0 tx=0).stop 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 -- 172.21.15.67:0/2348290940 shutdown_connections 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 --2- 172.21.15.67:0/2348290940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc104bb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 -- 172.21.15.67:0/2348290940 >> 172.21.15.67:0/2348290940 conn(0x7fd0bc077ab0 msgr2=0x7fd0bc077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 -- 172.21.15.67:0/2348290940 shutdown_connections 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.989+0000 7fd0c183d700 1 -- 172.21.15.67:0/2348290940 wait complete. 2023-10-07T23:07:34.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.990+0000 7fd0c183d700 1 Processor -- start 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.990+0000 7fd0c183d700 1 -- start start 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.990+0000 7fd0c183d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc1129b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.990+0000 7fd0c183d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc112ef0 con 0x7fd0bc1047a0 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.991+0000 7fd0baffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc1129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.991+0000 7fd0baffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc1129b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54322/0 (socket says 172.21.15.67:54322) 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.991+0000 7fd0baffd700 1 -- 172.21.15.67:0/3753592973 learned_addr learned my addr 172.21.15.67:0/3753592973 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.991+0000 7fd0baffd700 1 -- 172.21.15.67:0/3753592973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0094f0 con 0x7fd0bc1047a0 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0baffd700 1 --2- 172.21.15.67:0/3753592973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc1129b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac003080 tx=0x7fd0ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac01c460 con 0x7fd0bc1047a0 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac01c5c0 con 0x7fd0bc1047a0 2023-10-07T23:07:34.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ac01c890 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc113150 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc113d90 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd0ac01a450 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.992+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd0ac02e790 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.993+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc05f270 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:33.996+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd0ac01cd80 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.072+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd0bc04bb80 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.073+0000 7fd0a37fe700 1 -- 172.21.15.67:0/3753592973 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fd0ac01a890 con 0x7fd0bc1047a0 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.073+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 msgr2=0x7fd0bc1129b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.073+0000 7fd0c183d700 1 --2- 172.21.15.67:0/3753592973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc1129b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac003080 tx=0x7fd0ac016000 comp rx=0 tx=0).stop 2023-10-07T23:07:34.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.073+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 shutdown_connections 2023-10-07T23:07:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.073+0000 7fd0c183d700 1 --2- 172.21.15.67:0/3753592973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0bc1047a0 0x7fd0bc1129b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.073+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 >> 172.21.15.67:0/3753592973 conn(0x7fd0bc077ab0 msgr2=0x7fd0bc107280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.074+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 shutdown_connections 2023-10-07T23:07:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:34.074+0000 7fd0c183d700 1 -- 172.21.15.67:0/3753592973 wait complete. 2023-10-07T23:07:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2023-10-07T23:07:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2023-10-07T23:07:35.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:35.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2023-10-07T23:07:35.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:35.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2023-10-07T23:07:35.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-07T23:07:35.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-07T23:07:35.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.309+0000 7fa27c738700 1 Processor -- start 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.309+0000 7fa27c738700 1 -- start start 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.310+0000 7fa27c738700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa274104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.310+0000 7fa27c738700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274105180 con 0x7fa2741047a0 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.310+0000 7fa27a4d4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa274104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.310+0000 7fa27a4d4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa274104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54332/0 (socket says 172.21.15.67:54332) 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.310+0000 7fa27a4d4700 1 -- 172.21.15.67:0/3256582592 learned_addr learned my addr 172.21.15.67:0/3256582592 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.311+0000 7fa27a4d4700 1 -- 172.21.15.67:0/3256582592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2741052c0 con 0x7fa2741047a0 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.311+0000 7fa27a4d4700 1 --2- 172.21.15.67:0/3256582592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa274104bb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3714025d292c23d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.311+0000 7fa2794d2700 1 -- 172.21.15.67:0/3256582592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa270009e80 con 0x7fa2741047a0 2023-10-07T23:07:35.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.311+0000 7fa2794d2700 1 -- 172.21.15.67:0/3256582592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27000b860 con 0x7fa2741047a0 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.311+0000 7fa2794d2700 1 -- 172.21.15.67:0/3256582592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27000b9d0 con 0x7fa2741047a0 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 -- 172.21.15.67:0/3256582592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 msgr2=0x7fa274104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 --2- 172.21.15.67:0/3256582592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa274104bb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270009250 comp rx=0 tx=0).stop 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 -- 172.21.15.67:0/3256582592 shutdown_connections 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 --2- 172.21.15.67:0/3256582592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa274104bb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 -- 172.21.15.67:0/3256582592 >> 172.21.15.67:0/3256582592 conn(0x7fa274077ab0 msgr2=0x7fa274077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 -- 172.21.15.67:0/3256582592 shutdown_connections 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.312+0000 7fa27c738700 1 -- 172.21.15.67:0/3256582592 wait complete. 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27c738700 1 Processor -- start 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27c738700 1 -- start start 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27c738700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa2741129b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:35.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27c738700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274112ef0 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27a4d4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa2741129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27a4d4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa2741129b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54342/0 (socket says 172.21.15.67:54342) 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.313+0000 7fa27a4d4700 1 -- 172.21.15.67:0/4034836028 learned_addr learned my addr 172.21.15.67:0/4034836028 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa27a4d4700 1 -- 172.21.15.67:0/4034836028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700094f0 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa27a4d4700 1 --2- 172.21.15.67:0/4034836028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa2741129b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa270003080 tx=0x7fa270016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27001c460 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27001c5c0 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27001c890 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa274113150 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.314+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274113d90 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.315+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa27001a450 con 0x7fa2741047a0 2023-10-07T23:07:35.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.315+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa27002e790 con 0x7fa2741047a0 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.315+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27405f270 con 0x7fa2741047a0 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.318+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa27001cd80 con 0x7fa2741047a0 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.391+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa27410c210 con 0x7fa2741047a0 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.395+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa27002e950 con 0x7fa2741047a0 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.395+0000 7fa26affd700 1 -- 172.21.15.67:0/4034836028 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fa27001b630 con 0x7fa2741047a0 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 msgr2=0x7fa2741129b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 --2- 172.21.15.67:0/4034836028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa2741129b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa270003080 tx=0x7fa270016000 comp rx=0 tx=0).stop 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 shutdown_connections 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 --2- 172.21.15.67:0/4034836028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2741047a0 0x7fa2741129b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 >> 172.21.15.67:0/4034836028 conn(0x7fa274077ab0 msgr2=0x7fa274107280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:35.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 shutdown_connections 2023-10-07T23:07:35.767 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:35.396+0000 7fa27c738700 1 -- 172.21.15.67:0/4034836028 wait complete. 2023-10-07T23:07:35.767 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2023-10-07T23:07:35.767 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:35 smithi067 ceph-mon[81801]: from='client.? 172.21.15.67:0/4034836028' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-07T23:07:35.767 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:35 smithi067 ceph-mon[81801]: from='client.? 172.21.15.67:0/4034836028' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-07T23:07:36.715 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:36 smithi067 ceph-mon[81801]: from='client.? 172.21.15.67:0/3407393740' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:07:36.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.269+0000 7f594e9d6700 1 Processor -- start 2023-10-07T23:07:36.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.269+0000 7f594e9d6700 1 -- start start 2023-10-07T23:07:36.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.269+0000 7f594e9d6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f5948102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:36.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.269+0000 7f594e9d6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5948102c70 con 0x7f5948102320 2023-10-07T23:07:36.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.270+0000 7f5947fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f5948102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:36.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.270+0000 7f5947fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f5948102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54358/0 (socket says 172.21.15.67:54358) 2023-10-07T23:07:36.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.270+0000 7f5947fff700 1 -- 172.21.15.67:0/1577480394 learned_addr learned my addr 172.21.15.67:0/1577480394 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:36.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.270+0000 7f5947fff700 1 -- 172.21.15.67:0/1577480394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948102db0 con 0x7f5948102320 2023-10-07T23:07:36.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.270+0000 7f5947fff700 1 --2- 172.21.15.67:0/1577480394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f5948102730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59380098e0 tx=0x7f5938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fed904fcb86101f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:36.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.270+0000 7f5946ffd700 1 -- 172.21.15.67:0/1577480394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5938009e80 con 0x7f5948102320 2023-10-07T23:07:36.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f5946ffd700 1 -- 172.21.15.67:0/1577480394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f593800bd10 con 0x7f5948102320 2023-10-07T23:07:36.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f5946ffd700 1 -- 172.21.15.67:0/1577480394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5938011ce0 con 0x7f5948102320 2023-10-07T23:07:36.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 -- 172.21.15.67:0/1577480394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 msgr2=0x7f5948102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:36.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 --2- 172.21.15.67:0/1577480394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f5948102730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59380098e0 tx=0x7f5938009250 comp rx=0 tx=0).stop 2023-10-07T23:07:36.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 -- 172.21.15.67:0/1577480394 shutdown_connections 2023-10-07T23:07:36.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 --2- 172.21.15.67:0/1577480394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f5948102730 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:36.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 -- 172.21.15.67:0/1577480394 >> 172.21.15.67:0/1577480394 conn(0x7f59480fda80 msgr2=0x7f59480ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:36.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 -- 172.21.15.67:0/1577480394 shutdown_connections 2023-10-07T23:07:36.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.271+0000 7f594e9d6700 1 -- 172.21.15.67:0/1577480394 wait complete. 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.272+0000 7f594e9d6700 1 Processor -- start 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.272+0000 7f594e9d6700 1 -- start start 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f594e9d6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f59481131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f594e9d6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5948113730 con 0x7f5948102320 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f5947fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f59481131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f5947fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f59481131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54364/0 (socket says 172.21.15.67:54364) 2023-10-07T23:07:36.734 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f5947fff700 1 -- 172.21.15.67:0/3407393740 learned_addr learned my addr 172.21.15.67:0/3407393740 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f5947fff700 1 -- 172.21.15.67:0/3407393740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380094f0 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.273+0000 7f5947fff700 1 --2- 172.21.15.67:0/3407393740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f59481131f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5938006b00 tx=0x7f5938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f593801a640 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f593801ac50 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5938022bb0 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5948113930 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5948113d40 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f593802c440 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.274+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f593802b730 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.275+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594804bb80 con 0x7f5948102320 2023-10-07T23:07:36.735 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.278+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f593801adc0 con 0x7f5948102320 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.352+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f59481145d0 con 0x7f5948102320 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.352+0000 7f5944ff9700 1 -- 172.21.15.67:0/3407393740 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f5938022490 con 0x7f5948102320 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.353+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 msgr2=0x7f59481131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.353+0000 7f594e9d6700 1 --2- 172.21.15.67:0/3407393740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f59481131f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5938006b00 tx=0x7f5938009250 comp rx=0 tx=0).stop 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.353+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 shutdown_connections 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.353+0000 7f594e9d6700 1 --2- 172.21.15.67:0/3407393740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5948102320 0x7f59481131f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.353+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 >> 172.21.15.67:0/3407393740 conn(0x7f59480fda80 msgr2=0x7f59480fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.354+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 shutdown_connections 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:36.354+0000 7f594e9d6700 1 -- 172.21.15.67:0/3407393740 wait complete. 2023-10-07T23:07:36.736 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2023-10-07T23:07:37.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:36 smithi067 systemd[1]: Stopping Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:07:37.480 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:37 smithi067 conmon[81777]: 2023-10-07T23:07:37.182+0000 7f3acd0ab700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:07:37.480 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:37 smithi067 conmon[81777]: 2023-10-07T23:07:37.182+0000 7f3acd0ab700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-07T23:07:37.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:37 smithi067 bash[82335]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mon-a 2023-10-07T23:07:38.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:37 smithi067 systemd[1]: ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-07T23:07:38.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:37 smithi067 systemd[1]: Stopped Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:07:38.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:37 smithi067 systemd[1]: Starting Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:07:38.731 INFO:teuthology.orchestra.run.smithi067.stdout:Setting mon public_network to 172.21.0.0/20 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 bash[82659]: e8557d0be3149638c5c608a4dc2447fcbe7cc8082ee711215db007f8b4c4af93 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: set uid:gid to 167:167 (ceph:ceph) 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable), process ceph-mon, pid 7 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: pidfile_write: ignore empty --pid-file 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: load: jerasure load: lrc 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: RocksDB version: 6.15.5 2023-10-07T23:07:39.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Compile date Oct 6 2023 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: DB SUMMARY 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: DB Session ID: 7294MEV90PQSSJ9TLX3R 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: CURRENT file: CURRENT 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: IDENTITY file: IDENTITY 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 96364 ; 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.error_if_exists: 0 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.create_if_missing: 0 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.paranoid_checks: 1 2023-10-07T23:07:39.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.env: 0x55ad0be11a80 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.fs: Posix File System 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.info_log: 0x55ad0da79dc0 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_file_opening_threads: 16 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.statistics: (nil) 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.use_fsync: 0 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_log_file_size: 0 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.keep_log_file_num: 1000 2023-10-07T23:07:39.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.recycle_log_file_num: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.allow_fallocate: 1 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.allow_mmap_reads: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.allow_mmap_writes: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.use_direct_reads: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.create_missing_column_families: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.db_log_dir: 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-07T23:07:39.008 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.advise_random_on_open: 1 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.db_write_buffer_size: 0 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.write_buffer_manager: 0x55ad0e76c960 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.rate_limiter: (nil) 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.wal_recovery_mode: 2 2023-10-07T23:07:39.009 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.enable_thread_tracking: 0 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.enable_pipelined_write: 0 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.unordered_write: 0 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.row_cache: None 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.wal_filter: None 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.allow_ingest_behind: 0 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.preserve_deletes: 0 2023-10-07T23:07:39.010 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.two_write_queues: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.manual_wal_flush: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.atomic_flush: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.log_readahead_size: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.best_efforts_recovery: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.allow_data_in_errors: 0 2023-10-07T23:07:39.011 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.db_host_id: __hostname__ 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_background_jobs: 2 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_background_compactions: -1 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_subcompactions: 1 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_total_wal_size: 0 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-07T23:07:39.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_open_files: -1 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bytes_per_sync: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_readahead_size: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_background_flushes: -1 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Compression algorithms supported: 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kZSTD supported: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kXpressCompression supported: 0 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kLZ4HCCompression supported: 1 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kZlibCompression supported: 1 2023-10-07T23:07:39.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kSnappyCompression supported: 1 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kLZ4Compression supported: 1 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: kBZip2Compression supported: 0 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.merge_operator: 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_filter: None 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_filter_factory: None 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.sst_partitioner_factory: None 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-07T23:07:39.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ad0da4cf90) 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: hash_index_allow_collision: 1 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 1 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2023-10-07T23:07:39.015 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x55ad0da7f090 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2023-10-07T23:07:39.016 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 4 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.write_buffer_size: 33554432 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_write_buffer_number: 2 2023-10-07T23:07:39.017 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression: NoCompression 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression: Disabled 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.prefix_extractor: nullptr 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.num_levels: 7 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-07T23:07:39.018 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-07T23:07:39.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-07T23:07:39.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-07T23:07:39.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-07T23:07:39.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.level: 32767 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.strategy: 0 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compression_opts.enabled: false 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-07T23:07:39.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.target_file_size_base: 67108864 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-07T23:07:39.021 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.arena_block_size: 4194304 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.disable_auto_compactions: 0 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-07T23:07:39.022 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.table_properties_collectors: 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.inplace_update_support: 0 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-07T23:07:39.023 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.bloom_locality: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.max_successive_merges: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.paranoid_file_checks: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.force_consistency_checks: 1 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.report_bg_io_stats: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.ttl: 2592000 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.enable_blob_files: false 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.min_blob_size: 0 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.blob_file_size: 268435456 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-07T23:07:39.024 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720058701852, "job": 1, "event": "recovery_started", "wal_files": [10]} 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720058703543, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 91716, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 89811, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13142, "raw_average_key_size": 49, "raw_value_size": 82640, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 265, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1696720058, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "c59356e4-43ac-424a-b6cc-349b29fa4fd5", "db_session_id": "7294MEV90PQSSJ9TLX3R"}} 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720058704197, "job": 1, "event": "recovery_finished"} 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55ad0da13180 2023-10-07T23:07:39.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: DB pointer 0x55ad0dae8000 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.026 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 91.33 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 91.33 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:07:39.027 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 13.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 13.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:07:39.028 INFO:journalctl@ceph.mon.a.smithi067.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 91.33 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 91.33 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.3 0.00 0.00 1 0.001 0 0 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-07T23:07:39.029 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 13.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???) e1 preinit fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).mds e1 new map 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).mds e1 print_map 2023-10-07T23:07:39.030 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 systemd[1]: Started Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-07T23:07:39.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} 2023-10-07T23:07:39.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: fsmap 2023-10-07T23:07:39.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: osdmap e1: 0 total, 0 up, 0 in 2023-10-07T23:07:39.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:38 smithi067 ceph-mon[82723]: mgrmap e1: no daemons active 2023-10-07T23:07:39.903 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.289+0000 7f8e0bb63700 1 Processor -- start 2023-10-07T23:07:39.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.289+0000 7f8e0bb63700 1 -- start start 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.290+0000 7f8e0bb63700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.290+0000 7f8e0bb63700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04102c70 con 0x7f8e04102320 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.290+0000 7f8e098ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.290+0000 7f8e098ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54374/0 (socket says 172.21.15.67:54374) 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.290+0000 7f8e098ff700 1 -- 172.21.15.67:0/3031675636 learned_addr learned my addr 172.21.15.67:0/3031675636 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.291+0000 7f8e098ff700 1 -- 172.21.15.67:0/3031675636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04102db0 con 0x7f8e04102320 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.291+0000 7f8e098ff700 1 --2- 172.21.15.67:0/3031675636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04102730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e000098e0 tx=0x7f8e00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8660a4de4c79e79 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.291+0000 7f8e088fd700 1 -- 172.21.15.67:0/3031675636 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e00009e80 con 0x7f8e04102320 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.291+0000 7f8e088fd700 1 -- 172.21.15.67:0/3031675636 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e0000bd10 con 0x7f8e04102320 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.291+0000 7f8e088fd700 1 -- 172.21.15.67:0/3031675636 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e00011ce0 con 0x7f8e04102320 2023-10-07T23:07:39.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 -- 172.21.15.67:0/3031675636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 msgr2=0x7f8e04102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 --2- 172.21.15.67:0/3031675636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04102730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8e000098e0 tx=0x7f8e00009250 comp rx=0 tx=0).stop 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 -- 172.21.15.67:0/3031675636 shutdown_connections 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 --2- 172.21.15.67:0/3031675636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04102730 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 -- 172.21.15.67:0/3031675636 >> 172.21.15.67:0/3031675636 conn(0x7f8e040fda80 msgr2=0x7f8e040ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 -- 172.21.15.67:0/3031675636 shutdown_connections 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.292+0000 7f8e0bb63700 1 -- 172.21.15.67:0/3031675636 wait complete. 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.293+0000 7f8e0bb63700 1 Processor -- start 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.293+0000 7f8e0bb63700 1 -- start start 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.293+0000 7f8e0bb63700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.293+0000 7f8e0bb63700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04113690 con 0x7f8e04102320 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8e098ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8e098ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04113150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54380/0 (socket says 172.21.15.67:54380) 2023-10-07T23:07:39.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8e098ff700 1 -- 172.21.15.67:0/2134230974 learned_addr learned my addr 172.21.15.67:0/2134230974 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8e098ff700 1 -- 172.21.15.67:0/2134230974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000094f0 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8e098ff700 1 --2- 172.21.15.67:0/2134230974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04113150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006b00 tx=0x7f8e00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e0001a640 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e0001ac50 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e04113890 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.294+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e00022bb0 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.295+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e04113ca0 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.295+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8e0002c440 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.295+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8e0002b730 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.296+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0404bb80 con 0x7f8e04102320 2023-10-07T23:07:39.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.298+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8e0001adc0 con 0x7f8e04102320 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.371+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8e04104170 con 0x7f8e04102320 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.374+0000 7f8dfa7fc700 1 -- 172.21.15.67:0/2134230974 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8e00022490 con 0x7f8e04102320 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.374+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 msgr2=0x7f8e04113150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.374+0000 7f8e0bb63700 1 --2- 172.21.15.67:0/2134230974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04113150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006b00 tx=0x7f8e00009250 comp rx=0 tx=0).stop 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.375+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 shutdown_connections 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.375+0000 7f8e0bb63700 1 --2- 172.21.15.67:0/2134230974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e04102320 0x7f8e04113150 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.375+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 >> 172.21.15.67:0/2134230974 conn(0x7f8e040fda80 msgr2=0x7f8e040fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.375+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 shutdown_connections 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:39.375+0000 7f8e0bb63700 1 -- 172.21.15.67:0/2134230974 wait complete. 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2023-10-07T23:07:39.911 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 9283 ... 2023-10-07T23:07:40.206 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x 2023-10-07T23:07:40.206 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-07T23:07:40.418 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-227de690-6566-11ee-8db5-212e2dc638e7.target.wants/ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-227de690-6566-11ee-8db5-212e2dc638e7@.service. 2023-10-07T23:07:40.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:40 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2134230974' entity='client.admin' 2023-10-07T23:07:40.505 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:40 smithi067 systemd[1]: Starting Ceph mgr.x for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:07:41.313 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-07T23:07:41.313 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2023-10-07T23:07:41.331 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-07T23:07:41.331 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2023-10-07T23:07:41.331 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2023-10-07T23:07:41.331 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2023-10-07T23:07:41.349 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-07T23:07:41.349 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2023-10-07T23:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-07T23:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2023-10-07T23:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2023-10-07T23:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2023-10-07T23:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2023-10-07T23:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2023-10-07T23:07:41.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:41 smithi067 bash[83099]: ce6eb291ec33da823a9c5ce41391403f9c3a0fc352a5f942a2bb01052b82405e 2023-10-07T23:07:41.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:41 smithi067 systemd[1]: Started Ceph mgr.x for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:07:42.443 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-07T23:07:42.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:42.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-07T23:07:42.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-07T23:07:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2023-10-07T23:07:33.437121+0000", 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:42.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.021+0000 7f67e2bfc700 1 Processor -- start 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.021+0000 7f67e2bfc700 1 -- start start 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.022+0000 7f67e2bfc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.022+0000 7f67e2bfc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc079130 con 0x7f67dc078d20 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.022+0000 7f67e0998700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.022+0000 7f67e0998700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47568/0 (socket says 172.21.15.67:47568) 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.022+0000 7f67e0998700 1 -- 172.21.15.67:0/1649921927 learned_addr learned my addr 172.21.15.67:0/1649921927 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.022+0000 7f67e0998700 1 -- 172.21.15.67:0/1649921927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc077770 con 0x7f67dc078d20 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.023+0000 7f67e0998700 1 --2- 172.21.15.67:0/1649921927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc077230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098e0 tx=0x7f67d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f62eff513661a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.023+0000 7f67db7fe700 1 -- 172.21.15.67:0/1649921927 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67d0009e80 con 0x7f67dc078d20 2023-10-07T23:07:42.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.023+0000 7f67db7fe700 1 -- 172.21.15.67:0/1649921927 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67d000bd10 con 0x7f67dc078d20 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.023+0000 7f67db7fe700 1 -- 172.21.15.67:0/1649921927 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67d0011e20 con 0x7f67dc078d20 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 -- 172.21.15.67:0/1649921927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 msgr2=0x7f67dc077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 --2- 172.21.15.67:0/1649921927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc077230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098e0 tx=0x7f67d0009250 comp rx=0 tx=0).stop 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 -- 172.21.15.67:0/1649921927 shutdown_connections 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 --2- 172.21.15.67:0/1649921927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc077230 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 -- 172.21.15.67:0/1649921927 >> 172.21.15.67:0/1649921927 conn(0x7f67dc0fda80 msgr2=0x7f67dc0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 -- 172.21.15.67:0/1649921927 shutdown_connections 2023-10-07T23:07:42.461 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.024+0000 7f67e2bfc700 1 -- 172.21.15.67:0/1649921927 wait complete. 2023-10-07T23:07:42.461 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.025+0000 7f67e2bfc700 1 Processor -- start 2023-10-07T23:07:42.461 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.025+0000 7f67e2bfc700 1 -- start start 2023-10-07T23:07:42.461 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.025+0000 7f67e2bfc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc1177f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:42.461 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.025+0000 7f67e2bfc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc117d30 con 0x7f67dc078d20 2023-10-07T23:07:42.461 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.026+0000 7f67e0998700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc1177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.026+0000 7f67e0998700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc1177f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47584/0 (socket says 172.21.15.67:47584) 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.026+0000 7f67e0998700 1 -- 172.21.15.67:0/4205305214 learned_addr learned my addr 172.21.15.67:0/4205305214 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.026+0000 7f67e0998700 1 -- 172.21.15.67:0/4205305214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00094f0 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.026+0000 7f67e0998700 1 --2- 172.21.15.67:0/4205305214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc1177f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67d0011a10 tx=0x7f67d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.026+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67d0019600 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.027+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67d0019c10 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.027+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67d0010ab0 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.027+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc117f30 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.027+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc118340 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.027+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f67d0024430 con 0x7f67dc078d20 2023-10-07T23:07:42.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.027+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f67d002c6a0 con 0x7f67dc078d20 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.028+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc04bb80 con 0x7f67dc078d20 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.030+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f67d0029060 con 0x7f67dc078d20 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.106+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f67dc110be0 con 0x7f67dc078d20 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.107+0000 7f67d97fa700 1 -- 172.21.15.67:0/4205305214 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f67d0019d80 con 0x7f67dc078d20 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 msgr2=0x7f67dc1177f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 --2- 172.21.15.67:0/4205305214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc1177f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67d0011a10 tx=0x7f67d0009250 comp rx=0 tx=0).stop 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 shutdown_connections 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 --2- 172.21.15.67:0/4205305214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67dc078d20 0x7f67dc1177f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 >> 172.21.15.67:0/4205305214 conn(0x7f67dc0fda80 msgr2=0x7f67dc0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 shutdown_connections 2023-10-07T23:07:42.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:42.108+0000 7f67e2bfc700 1 -- 172.21.15.67:0/4205305214 wait complete. 2023-10-07T23:07:42.464 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2023-10-07T23:07:42.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:42 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4205305214' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-07T23:07:43.189 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:42 smithi067 conmon[83139]: 2023-10-07T23:07:42.858+0000 7fa4159541c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:07:43.190 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:43 smithi067 conmon[83139]: 2023-10-07T23:07:43.071+0000 7fa4159541c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:07:43.464 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:43 smithi067 conmon[83139]: 2023-10-07T23:07:43.190+0000 7fa4159541c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:07:43.725 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:43 smithi067 conmon[83139]: 2023-10-07T23:07:43.465+0000 7fa4159541c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:07:44.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:43 smithi067 conmon[83139]: 2023-10-07T23:07:43.726+0000 7fa4159541c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:07:44.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:43 smithi067 conmon[83139]: 2023-10-07T23:07:43.924+0000 7fa4159541c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:07:44.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:44 smithi067 conmon[83139]: 2023-10-07T23:07:44.032+0000 7fa4159541c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:07:45.005 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:44 smithi067 conmon[83139]: 2023-10-07T23:07:44.878+0000 7fa4159541c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:07:45.482 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:45 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1935960914' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-07T23:07:45.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-07T23:07:45.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-07T23:07:45.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:45.549 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2023-10-07T23:07:33.437121+0000", 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-07T23:07:45.550 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:07:45.554 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.019+0000 7f9bb04fe700 1 Processor -- start 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.019+0000 7f9bb04fe700 1 -- start start 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.020+0000 7f9bb04fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba80771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.020+0000 7f9bb04fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80790d0 con 0x7f9ba8078cc0 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.020+0000 7f9bae29a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba80771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.020+0000 7f9bae29a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba80771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47592/0 (socket says 172.21.15.67:47592) 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.020+0000 7f9bae29a700 1 -- 172.21.15.67:0/3366325321 learned_addr learned my addr 172.21.15.67:0/3366325321 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.021+0000 7f9bae29a700 1 -- 172.21.15.67:0/3366325321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8077710 con 0x7f9ba8078cc0 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.021+0000 7f9bae29a700 1 --2- 172.21.15.67:0/3366325321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba80771d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40098e0 tx=0x7f9ba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4a860e7c27fe90 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.021+0000 7f9bad298700 1 -- 172.21.15.67:0/3366325321 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4009e80 con 0x7f9ba8078cc0 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.021+0000 7f9bad298700 1 -- 172.21.15.67:0/3366325321 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ba400bd10 con 0x7f9ba8078cc0 2023-10-07T23:07:45.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.021+0000 7f9bad298700 1 -- 172.21.15.67:0/3366325321 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4011ce0 con 0x7f9ba8078cc0 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 -- 172.21.15.67:0/3366325321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 msgr2=0x7f9ba80771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 --2- 172.21.15.67:0/3366325321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba80771d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40098e0 tx=0x7f9ba4009250 comp rx=0 tx=0).stop 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 -- 172.21.15.67:0/3366325321 shutdown_connections 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 --2- 172.21.15.67:0/3366325321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba80771d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 -- 172.21.15.67:0/3366325321 >> 172.21.15.67:0/3366325321 conn(0x7f9ba80fda80 msgr2=0x7f9ba80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 -- 172.21.15.67:0/3366325321 shutdown_connections 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.022+0000 7f9bb04fe700 1 -- 172.21.15.67:0/3366325321 wait complete. 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.023+0000 7f9bb04fe700 1 Processor -- start 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.023+0000 7f9bb04fe700 1 -- start start 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bb04fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba8113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bb04fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8113790 con 0x7f9ba8078cc0 2023-10-07T23:07:45.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bae29a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba8113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bae29a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba8113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47604/0 (socket says 172.21.15.67:47604) 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bae29a700 1 -- 172.21.15.67:0/1935960914 learned_addr learned my addr 172.21.15.67:0/1935960914 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bae29a700 1 -- 172.21.15.67:0/1935960914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40094f0 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.024+0000 7f9bae29a700 1 --2- 172.21.15.67:0/1935960914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba8113250 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4011970 tx=0x7f9ba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4009e80 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ba401aaf0 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba8113990 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4022a90 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba8113da0 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9ba4022c40 con 0x7f9ba8078cc0 2023-10-07T23:07:45.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.025+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9ba402b3e0 con 0x7f9ba8078cc0 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.026+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba804bb80 con 0x7f9ba8078cc0 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.029+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9ba4022400 con 0x7f9ba8078cc0 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.125+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9ba8114690 con 0x7f9ba8078cc0 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.125+0000 7f9b9effd700 1 -- 172.21.15.67:0/1935960914 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f9ba401ac60 con 0x7f9ba8078cc0 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.126+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 msgr2=0x7f9ba8113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.126+0000 7f9bb04fe700 1 --2- 172.21.15.67:0/1935960914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba8113250 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4011970 tx=0x7f9ba4009250 comp rx=0 tx=0).stop 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.126+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 shutdown_connections 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.126+0000 7f9bb04fe700 1 --2- 172.21.15.67:0/1935960914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ba8078cc0 0x7f9ba8113250 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.126+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 >> 172.21.15.67:0/1935960914 conn(0x7f9ba80fda80 msgr2=0x7f9ba80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.127+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 shutdown_connections 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:45.127+0000 7f9bb04fe700 1 -- 172.21.15.67:0/1935960914 wait complete. 2023-10-07T23:07:45.558 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2023-10-07T23:07:45.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:45 smithi067 conmon[83139]: 2023-10-07T23:07:45.549+0000 7fa4159541c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:07:45.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:45 smithi067 conmon[83139]: 2023-10-07T23:07:45.670+0000 7fa4159541c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:07:46.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:45 smithi067 conmon[83139]: 2023-10-07T23:07:45.795+0000 7fa4159541c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:07:46.933 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:46 smithi067 conmon[83139]: 2023-10-07T23:07:46.586+0000 7fa4159541c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T23:07:46.933 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:46 smithi067 conmon[83139]: 2023-10-07T23:07:46.809+0000 7fa4159541c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:07:47.230 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:46 smithi067 conmon[83139]: 2023-10-07T23:07:46.932+0000 7fa4159541c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:07:47.231 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:47 smithi067 conmon[83139]: 2023-10-07T23:07:47.108+0000 7fa4159541c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T23:07:47.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:47 smithi067 conmon[83139]: 2023-10-07T23:07:47.230+0000 7fa4159541c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:07:48.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:48 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2127662962' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-07T23:07:48.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:48 smithi067 conmon[83139]: 2023-10-07T23:07:48.402+0000 7fa4159541c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T23:07:48.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:48.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:07:48.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:07:48.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2023-10-07T23:07:48.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-07T23:07:48.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-07T23:07:48.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-07T23:07:48.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-07T23:07:48.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-07T23:07:48.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2023-10-07T23:07:48.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-10-07T23:07:48.642 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-07T23:07:48.643 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.643 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-07T23:07:48.643 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:48.643 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-07T23:07:48.643 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-07T23:07:48.651 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-07T23:07:48.652 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2023-10-07T23:07:33.437121+0000", 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-07T23:07:48.653 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.110+0000 7f403f606700 1 Processor -- start 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.110+0000 7f403f606700 1 -- start start 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.110+0000 7f403f606700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.110+0000 7f403f606700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038079130 con 0x7f4038078d20 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.111+0000 7f403d3a2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.111+0000 7f403d3a2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47608/0 (socket says 172.21.15.67:47608) 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.111+0000 7f403d3a2700 1 -- 172.21.15.67:0/956089202 learned_addr learned my addr 172.21.15.67:0/956089202 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.111+0000 7f403d3a2700 1 -- 172.21.15.67:0/956089202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038077770 con 0x7f4038078d20 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.111+0000 7f403d3a2700 1 --2- 172.21.15.67:0/956089202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038077230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40340098e0 tx=0x7f4034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420fb9f924ba07e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.112+0000 7f402ffff700 1 -- 172.21.15.67:0/956089202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4034009e80 con 0x7f4038078d20 2023-10-07T23:07:48.654 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.112+0000 7f402ffff700 1 -- 172.21.15.67:0/956089202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f403400bd10 con 0x7f4038078d20 2023-10-07T23:07:48.655 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.112+0000 7f402ffff700 1 -- 172.21.15.67:0/956089202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4034011e20 con 0x7f4038078d20 2023-10-07T23:07:48.655 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.112+0000 7f403f606700 1 -- 172.21.15.67:0/956089202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 msgr2=0x7f4038077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:48.655 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.112+0000 7f403f606700 1 --2- 172.21.15.67:0/956089202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038077230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40340098e0 tx=0x7f4034009250 comp rx=0 tx=0).stop 2023-10-07T23:07:48.655 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.113+0000 7f403f606700 1 -- 172.21.15.67:0/956089202 shutdown_connections 2023-10-07T23:07:48.655 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.113+0000 7f403f606700 1 --2- 172.21.15.67:0/956089202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038077230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:48.655 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.113+0000 7f403f606700 1 -- 172.21.15.67:0/956089202 >> 172.21.15.67:0/956089202 conn(0x7f40380fda80 msgr2=0x7f40380ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.113+0000 7f403f606700 1 -- 172.21.15.67:0/956089202 shutdown_connections 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.113+0000 7f403f606700 1 -- 172.21.15.67:0/956089202 wait complete. 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.114+0000 7f403f606700 1 Processor -- start 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.114+0000 7f403f606700 1 -- start start 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.115+0000 7f403f606700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038113380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.115+0000 7f403f606700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40381138c0 con 0x7f4038078d20 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.115+0000 7f403d3a2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038113380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.115+0000 7f403d3a2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038113380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47612/0 (socket says 172.21.15.67:47612) 2023-10-07T23:07:48.656 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.115+0000 7f403d3a2700 1 -- 172.21.15.67:0/2127662962 learned_addr learned my addr 172.21.15.67:0/2127662962 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.115+0000 7f403d3a2700 1 -- 172.21.15.67:0/2127662962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40340094f0 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f403d3a2700 1 --2- 172.21.15.67:0/2127662962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038113380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4034006fd0 tx=0x7f4034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40340117a0 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4034009e80 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4038113ac0 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f403401adc0 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4038113ed0 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f403402c440 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.116+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4034022750 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.117+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403804bb80 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.119+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f403401a430 con 0x7f4038078d20 2023-10-07T23:07:48.657 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.196+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f403810ce00 con 0x7f4038078d20 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.196+0000 7f402dffb700 1 -- 172.21.15.67:0/2127662962 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f403401a6a0 con 0x7f4038078d20 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.197+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 msgr2=0x7f4038113380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.197+0000 7f403f606700 1 --2- 172.21.15.67:0/2127662962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038113380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4034006fd0 tx=0x7f4034009250 comp rx=0 tx=0).stop 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.197+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 shutdown_connections 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.197+0000 7f403f606700 1 --2- 172.21.15.67:0/2127662962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4038078d20 0x7f4038113380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.197+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 >> 172.21.15.67:0/2127662962 conn(0x7f40380fda80 msgr2=0x7f40380ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.198+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 shutdown_connections 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:48.198+0000 7f403f606700 1 -- 172.21.15.67:0/2127662962 wait complete. 2023-10-07T23:07:48.658 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (3/15)... 2023-10-07T23:07:48.843 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:48 smithi067 conmon[83139]: 2023-10-07T23:07:48.524+0000 7fa4159541c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:07:48.843 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:48 smithi067 conmon[83139]: 2023-10-07T23:07:48.644+0000 7fa4159541c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:07:49.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:48 smithi067 conmon[83139]: 2023-10-07T23:07:48.842+0000 7fa4159541c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:07:49.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:48 smithi067 conmon[83139]: 2023-10-07T23:07:48.972+0000 7fa4159541c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:07:49.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:49 smithi067 conmon[83139]: 2023-10-07T23:07:49.473+0000 7fa4159541c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:07:49.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:49 smithi067 conmon[83139]: 2023-10-07T23:07:49.593+0000 7fa4159541c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:07:49.755 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:49 smithi067 conmon[83139]: 2023-10-07T23:07:49.712+0000 7fa4159541c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:49 smithi067 conmon[83139]: 2023-10-07T23:07:49.829+0000 7fa4159541c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: Activating manager daemon x 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: mgrmap e2: x(active, starting, since 0.00415538s) 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-07T23:07:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: Manager daemon x is now available 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' 2023-10-07T23:07:50.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:49 smithi067 ceph-mon[82723]: from='mgr.14100 172.21.15.67:0/2174250102' entity='mgr.x' 2023-10-07T23:07:51.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-07T23:07:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-07T23:07:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-07T23:07:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-07T23:07:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2023-10-07T23:07:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2023-10-07T23:07:33.437121+0000", 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2023-10-07T23:07:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.165+0000 7f7ed2edc700 1 Processor -- start 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.165+0000 7f7ed2edc700 1 -- start start 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.166+0000 7f7ed2edc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc0771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.166+0000 7f7ed2edc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0790d0 con 0x7f7ecc078cc0 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.166+0000 7f7ed0c78700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc0771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.166+0000 7f7ed0c78700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc0771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47696/0 (socket says 172.21.15.67:47696) 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.166+0000 7f7ed0c78700 1 -- 172.21.15.67:0/2375224483 learned_addr learned my addr 172.21.15.67:0/2375224483 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.166+0000 7f7ed0c78700 1 -- 172.21.15.67:0/2375224483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc077710 con 0x7f7ecc078cc0 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.167+0000 7f7ed0c78700 1 --2- 172.21.15.67:0/2375224483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc0771d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098e0 tx=0x7f7ec0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d79f7252eba63b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:51.840 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.167+0000 7f7ecb7fe700 1 -- 172.21.15.67:0/2375224483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0009e80 con 0x7f7ecc078cc0 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.167+0000 7f7ecb7fe700 1 -- 172.21.15.67:0/2375224483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ec000bd10 con 0x7f7ecc078cc0 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.167+0000 7f7ecb7fe700 1 -- 172.21.15.67:0/2375224483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0011ce0 con 0x7f7ecc078cc0 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.167+0000 7f7ed2edc700 1 -- 172.21.15.67:0/2375224483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 msgr2=0x7f7ecc0771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.167+0000 7f7ed2edc700 1 --2- 172.21.15.67:0/2375224483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc0771d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098e0 tx=0x7f7ec0009250 comp rx=0 tx=0).stop 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.168+0000 7f7ed2edc700 1 -- 172.21.15.67:0/2375224483 shutdown_connections 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.168+0000 7f7ed2edc700 1 --2- 172.21.15.67:0/2375224483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc0771d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.168+0000 7f7ed2edc700 1 -- 172.21.15.67:0/2375224483 >> 172.21.15.67:0/2375224483 conn(0x7f7ecc0fda80 msgr2=0x7f7ecc0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.168+0000 7f7ed2edc700 1 -- 172.21.15.67:0/2375224483 shutdown_connections 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.168+0000 7f7ed2edc700 1 -- 172.21.15.67:0/2375224483 wait complete. 2023-10-07T23:07:51.841 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed2edc700 1 Processor -- start 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed2edc700 1 -- start start 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed2edc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed2edc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc113790 con 0x7f7ecc078cc0 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed0c78700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed0c78700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47710/0 (socket says 172.21.15.67:47710) 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed0c78700 1 -- 172.21.15.67:0/443869843 learned_addr learned my addr 172.21.15.67:0/443869843 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.169+0000 7f7ed0c78700 1 -- 172.21.15.67:0/443869843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00094f0 con 0x7f7ecc078cc0 2023-10-07T23:07:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.170+0000 7f7ed0c78700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc113250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011970 tx=0x7f7ec0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.170+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0009e80 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.170+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ec001aaf0 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.170+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc113990 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.170+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0022a90 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.170+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc113da0 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.171+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 3) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f7ec0022400 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.171+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc04bb80 con 0x7f7ecc078cc0 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.172+0000 7f7ec97fa700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f7ebc0364a0 0x7f7ebc038950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:51.843 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.172+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7ec004a120 con 0x7f7ecc078cc0 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.172+0000 7f7ecbfff700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f7ebc0364a0 0x7f7ebc038950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.173+0000 7f7ecbfff700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f7ebc0364a0 0x7f7ebc038950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009990 tx=0x7f7eb8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.178+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7ec0006d70 con 0x7f7ecc078cc0 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.459+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7ecc114a10 con 0x7f7ecc078cc0 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.460+0000 7f7ec97fa700 1 -- 172.21.15.67:0/443869843 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f7ec00213a0 con 0x7f7ecc078cc0 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f7ebc0364a0 msgr2=0x7f7ebc038950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f7ebc0364a0 0x7f7ebc038950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009990 tx=0x7f7eb8006e40 comp rx=0 tx=0).stop 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 msgr2=0x7f7ecc113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc113250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011970 tx=0x7f7ec0009250 comp rx=0 tx=0).stop 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 shutdown_connections 2023-10-07T23:07:51.844 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f7ebc0364a0 0x7f7ebc038950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:51.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 --2- 172.21.15.67:0/443869843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ecc078cc0 0x7f7ecc113250 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:51.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.462+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 >> 172.21.15.67:0/443869843 conn(0x7f7ecc0fda80 msgr2=0x7f7ecc0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:51.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.463+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 shutdown_connections 2023-10-07T23:07:51.845 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:51.463+0000 7f7ed2edc700 1 -- 172.21.15.67:0/443869843 wait complete. 2023-10-07T23:07:51.845 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2023-10-07T23:07:52.073 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:51 smithi067 ceph-mon[82723]: mgrmap e3: x(active, since 1.00844s) 2023-10-07T23:07:52.073 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:51 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/443869843' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-07T23:07:52.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:52.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2023-10-07T23:07:52.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.325+0000 7f819bf11700 1 Processor -- start 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.325+0000 7f819bf11700 1 -- start start 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.325+0000 7f819bf11700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f8194107390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.325+0000 7f819bf11700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194102d80 con 0x7f8194076bc0 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.326+0000 7f8199cad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f8194107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.326+0000 7f8199cad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f8194107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46950/0 (socket says 172.21.15.67:46950) 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.326+0000 7f8199cad700 1 -- 172.21.15.67:0/1349027833 learned_addr learned my addr 172.21.15.67:0/1349027833 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.326+0000 7f8199cad700 1 -- 172.21.15.67:0/1349027833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81941078d0 con 0x7f8194076bc0 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.326+0000 7f8199cad700 1 --2- 172.21.15.67:0/1349027833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f8194107390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81900098e0 tx=0x7f8190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b04e4451077d81 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f8198cab700 1 -- 172.21.15.67:0/1349027833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190009e80 con 0x7f8194076bc0 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f8198cab700 1 -- 172.21.15.67:0/1349027833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f819000bd10 con 0x7f8194076bc0 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f8198cab700 1 -- 172.21.15.67:0/1349027833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190011ce0 con 0x7f8194076bc0 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 -- 172.21.15.67:0/1349027833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 msgr2=0x7f8194107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 --2- 172.21.15.67:0/1349027833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f8194107390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81900098e0 tx=0x7f8190009250 comp rx=0 tx=0).stop 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 -- 172.21.15.67:0/1349027833 shutdown_connections 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 --2- 172.21.15.67:0/1349027833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f8194107390 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 -- 172.21.15.67:0/1349027833 >> 172.21.15.67:0/1349027833 conn(0x7f81940fc980 msgr2=0x7f81940fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 -- 172.21.15.67:0/1349027833 shutdown_connections 2023-10-07T23:07:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.327+0000 7f819bf11700 1 -- 172.21.15.67:0/1349027833 wait complete. 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.328+0000 7f819bf11700 1 Processor -- start 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.328+0000 7f819bf11700 1 -- start start 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f819bf11700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f81941131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f819bf11700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194113730 con 0x7f8194076bc0 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f8199cad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f81941131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f8199cad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f81941131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46962/0 (socket says 172.21.15.67:46962) 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f8199cad700 1 -- 172.21.15.67:0/4064803945 learned_addr learned my addr 172.21.15.67:0/4064803945 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f8199cad700 1 -- 172.21.15.67:0/4064803945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900094f0 con 0x7f8194076bc0 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.329+0000 7f8199cad700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f81941131f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8190011970 tx=0x7f8190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.330+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190009e80 con 0x7f8194076bc0 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.330+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f819001aaf0 con 0x7f8194076bc0 2023-10-07T23:07:52.914 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.330+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8194113930 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.330+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8190022a90 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.330+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8194113d40 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.331+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 43254+0+0 (secure 0 0 0) 0x7f8190022c90 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.331+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819404bb80 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.331+0000 7f818a7fc700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f8180036570 0x7f8180038a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.331+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f819004a3c0 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.331+0000 7f81994ac700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f8180036570 0x7f8180038a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.332+0000 7f81994ac700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f8180036570 0x7f8180038a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8184009990 tx=0x7f8184006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.336+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8190006d70 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.537+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8194114790 con 0x7f8194076bc0 2023-10-07T23:07:52.915 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.539+0000 7f818a7fc700 1 -- 172.21.15.67:0/4064803945 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f81900213a0 con 0x7f8194076bc0 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f8180036570 msgr2=0x7f8180038a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f8180036570 0x7f8180038a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8184009990 tx=0x7f8184006e40 comp rx=0 tx=0).stop 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 msgr2=0x7f81941131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f81941131f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8190011970 tx=0x7f8190009250 comp rx=0 tx=0).stop 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 shutdown_connections 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f8180036570 0x7f8180038a20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 --2- 172.21.15.67:0/4064803945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8194076bc0 0x7f81941131f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 >> 172.21.15.67:0/4064803945 conn(0x7f81940fc980 msgr2=0x7f81940fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 shutdown_connections 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:52.541+0000 7f819bf11700 1 -- 172.21.15.67:0/4064803945 wait complete. 2023-10-07T23:07:52.916 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2023-10-07T23:07:53.193 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:52 smithi067 ceph-mon[82723]: mgrmap e4: x(active, since 2s) 2023-10-07T23:07:53.193 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:52 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4064803945' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-07T23:07:54.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:53 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3205950394' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-07T23:07:54.152 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:53 smithi067 conmon[83139]: ignoring --setuser ceph since I am not root 2023-10-07T23:07:54.153 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:53 smithi067 conmon[83139]: ignoring --setgroup ceph since I am not root 2023-10-07T23:07:54.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.441+0000 7f422a030700 1 Processor -- start 2023-10-07T23:07:54.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.442+0000 7f422a030700 1 -- start start 2023-10-07T23:07:54.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.442+0000 7f422a030700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42240771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:54.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.442+0000 7f422a030700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240790d0 con 0x7f4224078cc0 2023-10-07T23:07:54.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.442+0000 7f42237fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42240771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.442+0000 7f42237fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42240771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46978/0 (socket says 172.21.15.67:46978) 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.442+0000 7f42237fe700 1 -- 172.21.15.67:0/1141863622 learned_addr learned my addr 172.21.15.67:0/1141863622 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.443+0000 7f42237fe700 1 -- 172.21.15.67:0/1141863622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224077710 con 0x7f4224078cc0 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.443+0000 7f42237fe700 1 --2- 172.21.15.67:0/1141863622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42240771d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42140098e0 tx=0x7f4214009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2075d620aea32b7d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.443+0000 7f42227fc700 1 -- 172.21.15.67:0/1141863622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4214009e80 con 0x7f4224078cc0 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.443+0000 7f42227fc700 1 -- 172.21.15.67:0/1141863622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f421400bd10 con 0x7f4224078cc0 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.443+0000 7f42227fc700 1 -- 172.21.15.67:0/1141863622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4214011ce0 con 0x7f4224078cc0 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 -- 172.21.15.67:0/1141863622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 msgr2=0x7f42240771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 --2- 172.21.15.67:0/1141863622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42240771d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42140098e0 tx=0x7f4214009250 comp rx=0 tx=0).stop 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 -- 172.21.15.67:0/1141863622 shutdown_connections 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 --2- 172.21.15.67:0/1141863622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42240771d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 -- 172.21.15.67:0/1141863622 >> 172.21.15.67:0/1141863622 conn(0x7f42240fdaa0 msgr2=0x7f42240ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 -- 172.21.15.67:0/1141863622 shutdown_connections 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.444+0000 7f422a030700 1 -- 172.21.15.67:0/1141863622 wait complete. 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f422a030700 1 Processor -- start 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f422a030700 1 -- start start 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f422a030700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42241131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f422a030700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42241136f0 con 0x7f4224078cc0 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f42237fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42241131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f42237fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42241131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46988/0 (socket says 172.21.15.67:46988) 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.445+0000 7f42237fe700 1 -- 172.21.15.67:0/3205950394 learned_addr learned my addr 172.21.15.67:0/3205950394 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f42237fe700 1 -- 172.21.15.67:0/3205950394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140094f0 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f42237fe700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42241131b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4214011650 tx=0x7f4214009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f421401a640 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f421401ac50 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42241138f0 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4214022b20 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.446+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4224113d00 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.447+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 43254+0+0 (secure 0 0 0) 0x7f4214022d10 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.447+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422404bb80 con 0x7f4224078cc0 2023-10-07T23:07:54.387 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.448+0000 7f420bfff700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f420c0365b0 0x7f420c038a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.448+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f421404a470 con 0x7f4224078cc0 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.448+0000 7f4222ffd700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f420c0365b0 0x7f420c038a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.449+0000 7f4222ffd700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f420c0365b0 0x7f420c038a60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4218009990 tx=0x7f4218006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.453+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4214021580 con 0x7f4224078cc0 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.709+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f4224114950 con 0x7f4224078cc0 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.849+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 5) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f42140224c0 con 0x7f4224078cc0 2023-10-07T23:07:54.388 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.850+0000 7f420bfff700 1 -- 172.21.15.67:0/3205950394 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f4214028900 con 0x7f4224078cc0 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f420c0365b0 msgr2=0x7f420c038a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f420c0365b0 0x7f420c038a60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4218009990 tx=0x7f4218006e40 comp rx=0 tx=0).stop 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 msgr2=0x7f42241131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42241131b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4214011650 tx=0x7f4214009250 comp rx=0 tx=0).stop 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 shutdown_connections 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f420c0365b0 0x7f420c038a60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 --2- 172.21.15.67:0/3205950394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4224078cc0 0x7f42241131b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 >> 172.21.15.67:0/3205950394 conn(0x7f42240fdaa0 msgr2=0x7f42240fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.852+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 shutdown_connections 2023-10-07T23:07:54.389 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:53.853+0000 7f422a030700 1 -- 172.21.15.67:0/3205950394 wait complete. 2023-10-07T23:07:55.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3205950394' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-07T23:07:55.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:54 smithi067 ceph-mon[82723]: mgrmap e5: x(active, since 4s) 2023-10-07T23:07:55.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:07:55.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-07T23:07:55.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-07T23:07:55.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-07T23:07:55.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-07T23:07:55.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:07:55.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.872+0000 7f27fa0c2700 1 Processor -- start 2023-10-07T23:07:55.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.872+0000 7f27fa0c2700 1 -- start start 2023-10-07T23:07:55.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.872+0000 7f27fa0c2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f4104f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:55.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.872+0000 7f27fa0c2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f41054b0 con 0x7f27f4076be0 2023-10-07T23:07:55.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.873+0000 7f27f37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f4104f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:55.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.873+0000 7f27f37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f4104f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47024/0 (socket says 172.21.15.67:47024) 2023-10-07T23:07:55.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.873+0000 7f27f37fe700 1 -- 172.21.15.67:0/3563978012 learned_addr learned my addr 172.21.15.67:0/3563978012 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:55.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.873+0000 7f27f37fe700 1 -- 172.21.15.67:0/3563978012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f41055f0 con 0x7f27f4076be0 2023-10-07T23:07:55.467 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.873+0000 7f27f37fe700 1 --2- 172.21.15.67:0/3563978012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f4104f70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27e40098e0 tx=0x7f27e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7e8417b02ad4cb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:55.467 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27f27fc700 1 -- 172.21.15.67:0/3563978012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27e4009e80 con 0x7f27f4076be0 2023-10-07T23:07:55.467 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27f27fc700 1 -- 172.21.15.67:0/3563978012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27e400bd10 con 0x7f27f4076be0 2023-10-07T23:07:55.467 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27f27fc700 1 -- 172.21.15.67:0/3563978012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27e4011ce0 con 0x7f27f4076be0 2023-10-07T23:07:55.467 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3563978012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 msgr2=0x7f27f4104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:55.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 --2- 172.21.15.67:0/3563978012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f4104f70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27e40098e0 tx=0x7f27e4009250 comp rx=0 tx=0).stop 2023-10-07T23:07:55.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3563978012 shutdown_connections 2023-10-07T23:07:55.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 --2- 172.21.15.67:0/3563978012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f4104f70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:55.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3563978012 >> 172.21.15.67:0/3563978012 conn(0x7f27f40fc980 msgr2=0x7f27f40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:55.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3563978012 shutdown_connections 2023-10-07T23:07:55.468 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.874+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3563978012 wait complete. 2023-10-07T23:07:55.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.875+0000 7f27fa0c2700 1 Processor -- start 2023-10-07T23:07:55.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.875+0000 7f27fa0c2700 1 -- start start 2023-10-07T23:07:55.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.876+0000 7f27fa0c2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f41029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:55.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.876+0000 7f27fa0c2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f4100e70 con 0x7f27f4076be0 2023-10-07T23:07:55.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.876+0000 7f27f37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f41029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:07:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.876+0000 7f27f37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f41029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47030/0 (socket says 172.21.15.67:47030) 2023-10-07T23:07:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.876+0000 7f27f37fe700 1 -- 172.21.15.67:0/3836389423 learned_addr learned my addr 172.21.15.67:0/3836389423 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:07:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.876+0000 7f27f37fe700 1 -- 172.21.15.67:0/3836389423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40094f0 con 0x7f27f4076be0 2023-10-07T23:07:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.877+0000 7f27f37fe700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f41029f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27e4011970 tx=0x7f27e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:07:55.470 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.877+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27e4009e80 con 0x7f27f4076be0 2023-10-07T23:07:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.877+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27e401aaf0 con 0x7f27f4076be0 2023-10-07T23:07:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.877+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f4101070 con 0x7f27f4076be0 2023-10-07T23:07:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.877+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27e4022a90 con 0x7f27f4076be0 2023-10-07T23:07:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.877+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f4101480 con 0x7f27f4076be0 2023-10-07T23:07:55.471 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.878+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f27e4022c90 con 0x7f27f4076be0 2023-10-07T23:07:55.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.878+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f404bb80 con 0x7f27f4076be0 2023-10-07T23:07:55.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.879+0000 7f27dbfff700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 0x7f27dc038a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:07:55.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.879+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f27e404a590 con 0x7f27f4076be0 2023-10-07T23:07:55.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.879+0000 7f27f2ffd700 1 -- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 msgr2=0x7f27dc038a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:07:55.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.879+0000 7f27f2ffd700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 0x7f27dc038a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:07:55.472 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:54.887+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f27e4006d70 con 0x7f27f4076be0 2023-10-07T23:07:55.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.080+0000 7f27f2ffd700 1 -- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 msgr2=0x7f27dc038a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:07:55.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.080+0000 7f27f2ffd700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 0x7f27dc038a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:07:55.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.149+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f27f4101b90 con 0x7f27f4076be0 2023-10-07T23:07:55.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.149+0000 7f27dbfff700 1 -- 172.21.15.67:0/3836389423 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f27e40213a0 con 0x7f27f4076be0 2023-10-07T23:07:55.473 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.151+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 msgr2=0x7f27dc038a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:07:55.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.151+0000 7f27fa0c2700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 0x7f27dc038a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:55.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.151+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 msgr2=0x7f27f41029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:07:55.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.151+0000 7f27fa0c2700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f41029f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27e4011970 tx=0x7f27e4009250 comp rx=0 tx=0).stop 2023-10-07T23:07:55.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.152+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 shutdown_connections 2023-10-07T23:07:55.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.152+0000 7f27fa0c2700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7f27dc0365b0 0x7f27dc038a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:55.474 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.152+0000 7f27fa0c2700 1 --2- 172.21.15.67:0/3836389423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f4076be0 0x7f27f41029f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:07:55.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.152+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 >> 172.21.15.67:0/3836389423 conn(0x7f27f40fc980 msgr2=0x7f27f40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:07:55.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.152+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 shutdown_connections 2023-10-07T23:07:55.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:55.152+0000 7f27fa0c2700 1 -- 172.21.15.67:0/3836389423 wait complete. 2023-10-07T23:07:55.475 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2023-10-07T23:07:55.475 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2023-10-07T23:07:55.774 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:55 smithi067 conmon[83139]: 2023-10-07T23:07:55.553+0000 7fd7e5b521c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:07:56.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:07:55 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3836389423' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-07T23:07:56.229 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:55 smithi067 conmon[83139]: 2023-10-07T23:07:55.773+0000 7fd7e5b521c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:07:56.229 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:55 smithi067 conmon[83139]: 2023-10-07T23:07:55.899+0000 7fd7e5b521c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:07:56.443 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:56 smithi067 conmon[83139]: 2023-10-07T23:07:56.179+0000 7fd7e5b521c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:07:56.748 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:56 smithi067 conmon[83139]: 2023-10-07T23:07:56.442+0000 7fd7e5b521c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:07:56.748 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:56 smithi067 conmon[83139]: 2023-10-07T23:07:56.639+0000 7fd7e5b521c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:07:57.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:56 smithi067 conmon[83139]: 2023-10-07T23:07:56.747+0000 7fd7e5b521c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:07:58.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:57 smithi067 conmon[83139]: 2023-10-07T23:07:57.530+0000 7fd7e5b521c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:07:58.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:58 smithi067 conmon[83139]: 2023-10-07T23:07:58.150+0000 7fd7e5b521c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:07:58.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:58 smithi067 conmon[83139]: 2023-10-07T23:07:58.273+0000 7fd7e5b521c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:07:58.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:58 smithi067 conmon[83139]: 2023-10-07T23:07:58.396+0000 7fd7e5b521c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:07:59.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:59 smithi067 conmon[83139]: 2023-10-07T23:07:59.184+0000 7fd7e5b521c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T23:07:59.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:59 smithi067 conmon[83139]: 2023-10-07T23:07:59.405+0000 7fd7e5b521c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:07:59.822 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:59 smithi067 conmon[83139]: 2023-10-07T23:07:59.527+0000 7fd7e5b521c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:07:59.823 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:59 smithi067 conmon[83139]: 2023-10-07T23:07:59.702+0000 7fd7e5b521c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T23:08:00.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:07:59 smithi067 conmon[83139]: 2023-10-07T23:07:59.822+0000 7fd7e5b521c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:08:01.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:00 smithi067 conmon[83139]: 2023-10-07T23:08:00.937+0000 7fd7e5b521c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T23:08:01.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:01 smithi067 conmon[83139]: 2023-10-07T23:08:01.053+0000 7fd7e5b521c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:08:01.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:01 smithi067 conmon[83139]: 2023-10-07T23:08:01.171+0000 7fd7e5b521c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:08:01.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:01 smithi067 conmon[83139]: 2023-10-07T23:08:01.368+0000 7fd7e5b521c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:08:01.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:01 smithi067 conmon[83139]: 2023-10-07T23:08:01.498+0000 7fd7e5b521c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:08:02.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:01 smithi067 conmon[83139]: 2023-10-07T23:08:01.985+0000 7fd7e5b521c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:08:02.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:02 smithi067 conmon[83139]: 2023-10-07T23:08:02.106+0000 7fd7e5b521c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:08:02.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:02 smithi067 conmon[83139]: 2023-10-07T23:08:02.225+0000 7fd7e5b521c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:08:02.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:02 smithi067 conmon[83139]: 2023-10-07T23:08:02.342+0000 7fd7e5b521c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:08:02.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: Active manager daemon x restarted 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: Activating manager daemon x 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: osdmap e2: 0 total, 0 up, 0 in 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: mgrmap e6: x(active, starting, since 0.00347203s) 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: Manager daemon x is now available 2023-10-07T23:08:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:02.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:02.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:02.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:02.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:02 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:03.724 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:03 smithi067 ceph-mon[82723]: Found migration_current of "None". Setting to last migration. 2023-10-07T23:08:03.724 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:03 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-07T23:08:03.724 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:03 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-07T23:08:03.724 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:03 smithi067 ceph-mon[82723]: mgrmap e7: x(active, since 1.00686s) 2023-10-07T23:08:03.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.014+0000 7fc9c3900700 1 Processor -- start 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.014+0000 7fc9c3900700 1 -- start start 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.015+0000 7fc9c3900700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.015+0000 7fc9c3900700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0790b0 con 0x7fc9bc078ca0 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.015+0000 7fc9c169c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.015+0000 7fc9c169c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc0771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47042/0 (socket says 172.21.15.67:47042) 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.015+0000 7fc9c169c700 1 -- 172.21.15.67:0/1469611372 learned_addr learned my addr 172.21.15.67:0/1469611372 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:03.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.016+0000 7fc9c169c700 1 -- 172.21.15.67:0/1469611372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0776f0 con 0x7fc9bc078ca0 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.016+0000 7fc9c169c700 1 --2- 172.21.15.67:0/1469611372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc0771b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80098e0 tx=0x7fc9b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aca1373a65f5f42 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.016+0000 7fc9b3fff700 1 -- 172.21.15.67:0/1469611372 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8009e80 con 0x7fc9bc078ca0 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.016+0000 7fc9b3fff700 1 -- 172.21.15.67:0/1469611372 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9b800bd10 con 0x7fc9bc078ca0 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.016+0000 7fc9b3fff700 1 -- 172.21.15.67:0/1469611372 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8011ce0 con 0x7fc9bc078ca0 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 -- 172.21.15.67:0/1469611372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 msgr2=0x7fc9bc0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 --2- 172.21.15.67:0/1469611372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc0771b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80098e0 tx=0x7fc9b8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 -- 172.21.15.67:0/1469611372 shutdown_connections 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 --2- 172.21.15.67:0/1469611372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc0771b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 -- 172.21.15.67:0/1469611372 >> 172.21.15.67:0/1469611372 conn(0x7fc9bc0fda80 msgr2=0x7fc9bc0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 -- 172.21.15.67:0/1469611372 shutdown_connections 2023-10-07T23:08:03.754 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.017+0000 7fc9c3900700 1 -- 172.21.15.67:0/1469611372 wait complete. 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.018+0000 7fc9c3900700 1 Processor -- start 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.018+0000 7fc9c3900700 1 -- start start 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.018+0000 7fc9c3900700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.018+0000 7fc9c3900700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc113780 con 0x7fc9bc078ca0 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9c169c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9c169c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47052/0 (socket says 172.21.15.67:47052) 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9c169c700 1 -- 172.21.15.67:0/4144399248 learned_addr learned my addr 172.21.15.67:0/4144399248 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9c169c700 1 -- 172.21.15.67:0/4144399248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b80094f0 con 0x7fc9bc078ca0 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9c169c700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc113240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011970 tx=0x7fc9b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8009e80 con 0x7fc9bc078ca0 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9b801aaf0 con 0x7fc9bc078ca0 2023-10-07T23:08:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.019+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8022a90 con 0x7fc9bc078ca0 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.020+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc113980 con 0x7fc9bc078ca0 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.020+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc113d90 con 0x7fc9bc078ca0 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.021+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 43265+0+0 (secure 0 0 0) 0x7fc9b8022c40 con 0x7fc9bc078ca0 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.021+0000 7fc9b1ffb700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.021+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 --> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9a80391d0 con 0x7fc9a8036610 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.021+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc9b804a570 con 0x7fc9bc078ca0 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.021+0000 7fc9c0e9b700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.021+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.222+0000 7fc9c0e9b700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.222+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.623+0000 7fc9c0e9b700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:08:03.756 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:56.623+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:57.424+0000 7fc9c0e9b700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:57.424+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:59.025+0000 7fc9c0e9b700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:07:59.025+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:02.228+0000 7fc9c0e9b700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3941151123 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:02.228+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:02.347+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 6) v1 ==== 43032+0+0 (secure 0 0 0) 0x7fc9b801ac60 con 0x7fc9bc078ca0 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:02.347+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 msgr2=0x7fc9a8038ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:02.347+0000 7fc9b1ffb700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.349+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 7) v1 ==== 43159+0+0 (secure 0 0 0) 0x7fc9b804c020 con 0x7fc9bc078ca0 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.349+0000 7fc9b1ffb700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc9a8039ee0 0x7fc9a803c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:03.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.349+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9a80391d0 con 0x7fc9a8039ee0 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.350+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc9a8039ee0 0x7fc9a803c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.351+0000 7fc9c0e9b700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc9a8039ee0 0x7fc9a803c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0039d0 tx=0x7fc9ac009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.352+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fc9a80391d0 con 0x7fc9a8039ee0 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.358+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc9bc04bb80 con 0x7fc9a8039ee0 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.358+0000 7fc9b1ffb700 1 -- 172.21.15.67:0/4144399248 <== mgr.14120 v2:172.21.15.67:6800/2928618291 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc9bc04bb80 con 0x7fc9a8039ee0 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc9a8039ee0 msgr2=0x7fc9a803c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc9a8039ee0 0x7fc9a803c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0039d0 tx=0x7fc9ac009350 comp rx=0 tx=0).stop 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 msgr2=0x7fc9bc113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc113240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011970 tx=0x7fc9b8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 shutdown_connections 2023-10-07T23:08:03.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc9a8039ee0 0x7fc9a803c2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9bc078ca0 0x7fc9bc113240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 --2- 172.21.15.67:0/4144399248 >> [v2:172.21.15.67:6800/3941151123,v1:172.21.15.67:6801/3941151123] conn(0x7fc9a8036610 0x7fc9a8038ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 >> 172.21.15.67:0/4144399248 conn(0x7fc9bc0fda80 msgr2=0x7fc9bc0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 shutdown_connections 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:03.359+0000 7fc9c3900700 1 -- 172.21.15.67:0/4144399248 wait complete. 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2023-10-07T23:08:03.759 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2023-10-07T23:08:05.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:04 smithi067 ceph-mon[82723]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-07T23:08:05.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:04 smithi067 ceph-mon[82723]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-07T23:08:05.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:04 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:05.127 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.429+0000 7f1a5453f700 1 Processor -- start 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.429+0000 7f1a5453f700 1 -- start start 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.429+0000 7f1a5453f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.429+0000 7f1a5453f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c102c70 con 0x7f1a4c102320 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.430+0000 7f1a522db700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.430+0000 7f1a522db700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52982/0 (socket says 172.21.15.67:52982) 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.430+0000 7f1a522db700 1 -- 172.21.15.67:0/3671059170 learned_addr learned my addr 172.21.15.67:0/3671059170 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.430+0000 7f1a522db700 1 -- 172.21.15.67:0/3671059170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c102db0 con 0x7f1a4c102320 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.430+0000 7f1a522db700 1 --2- 172.21.15.67:0/3671059170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c102730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098e0 tx=0x7f1a48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d46acdae0e3d10d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.430+0000 7f1a512d9700 1 -- 172.21.15.67:0/3671059170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a48009e80 con 0x7f1a4c102320 2023-10-07T23:08:05.128 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a512d9700 1 -- 172.21.15.67:0/3671059170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a4800bd10 con 0x7f1a4c102320 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a512d9700 1 -- 172.21.15.67:0/3671059170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a48011ce0 con 0x7f1a4c102320 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 -- 172.21.15.67:0/3671059170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 msgr2=0x7f1a4c102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 --2- 172.21.15.67:0/3671059170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c102730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098e0 tx=0x7f1a48009250 comp rx=0 tx=0).stop 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 -- 172.21.15.67:0/3671059170 shutdown_connections 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 --2- 172.21.15.67:0/3671059170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c102730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 -- 172.21.15.67:0/3671059170 >> 172.21.15.67:0/3671059170 conn(0x7f1a4c0fda80 msgr2=0x7f1a4c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 -- 172.21.15.67:0/3671059170 shutdown_connections 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.431+0000 7f1a5453f700 1 -- 172.21.15.67:0/3671059170 wait complete. 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.432+0000 7f1a5453f700 1 Processor -- start 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.432+0000 7f1a5453f700 1 -- start start 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.432+0000 7f1a5453f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c1131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:05.129 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.433+0000 7f1a5453f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c113700 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.433+0000 7f1a522db700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c1131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.433+0000 7f1a522db700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c1131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52994/0 (socket says 172.21.15.67:52994) 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.433+0000 7f1a522db700 1 -- 172.21.15.67:0/3984003294 learned_addr learned my addr 172.21.15.67:0/3984003294 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.433+0000 7f1a522db700 1 -- 172.21.15.67:0/3984003294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480094f0 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.433+0000 7f1a522db700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c1131c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a48006b00 tx=0x7f1a48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.434+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a4801a640 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.434+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c113900 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.434+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a4801ac50 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.434+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a48022bb0 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.434+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c113d10 con 0x7f1a4c102320 2023-10-07T23:08:05.130 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.435+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 7) v1 ==== 43159+0+0 (secure 0 0 0) 0x7f1a48022490 con 0x7f1a4c102320 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.435+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c04bb80 con 0x7f1a4c102320 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.435+0000 7f1a42ffd700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1a380364a0 0x7f1a38038950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.435+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1a4804a300 con 0x7f1a4c102320 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.436+0000 7f1a51ada700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1a380364a0 0x7f1a38038950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.436+0000 7f1a51ada700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1a380364a0 0x7f1a38038950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009990 tx=0x7f1a3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.441+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f1a4801adc0 con 0x7f1a4c102320 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.658+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a4c0fee60 con 0x7f1a380364a0 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.667+0000 7f1a42ffd700 1 -- 172.21.15.67:0/3984003294 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1a4c0fee60 con 0x7f1a380364a0 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1a380364a0 msgr2=0x7f1a38038950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1a380364a0 0x7f1a38038950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009990 tx=0x7f1a3c006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:05.131 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 msgr2=0x7f1a4c1131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c1131c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a48006b00 tx=0x7f1a48009250 comp rx=0 tx=0).stop 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 shutdown_connections 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1a380364a0 0x7f1a38038950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 --2- 172.21.15.67:0/3984003294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c102320 0x7f1a4c1131c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 >> 172.21.15.67:0/3984003294 conn(0x7f1a4c0fda80 msgr2=0x7f1a4c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 shutdown_connections 2023-10-07T23:08:05.132 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:04.670+0000 7f1a5453f700 1 -- 172.21.15.67:0/3984003294 wait complete. 2023-10-07T23:08:06.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:05 smithi067 ceph-mon[82723]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:06.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:05 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:06.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:05 smithi067 ceph-mon[82723]: mgrmap e8: x(active, since 2s) 2023-10-07T23:08:06.211 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-07T23:08:06.211 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.623+0000 7f26631cc700 1 Processor -- start 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.624+0000 7f26631cc700 1 -- start start 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.624+0000 7f26631cc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.624+0000 7f26631cc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0790b0 con 0x7f265c078ca0 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.624+0000 7f2660f68700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.624+0000 7f2660f68700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c0771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53010/0 (socket says 172.21.15.67:53010) 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.624+0000 7f2660f68700 1 -- 172.21.15.67:0/2715450864 learned_addr learned my addr 172.21.15.67:0/2715450864 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.625+0000 7f2660f68700 1 -- 172.21.15.67:0/2715450864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0776f0 con 0x7f265c078ca0 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.625+0000 7f2660f68700 1 --2- 172.21.15.67:0/2715450864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c0771b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26500098e0 tx=0x7f2650009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07de88df268a546 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.625+0000 7f265b7fe700 1 -- 172.21.15.67:0/2715450864 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2650009e80 con 0x7f265c078ca0 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.625+0000 7f265b7fe700 1 -- 172.21.15.67:0/2715450864 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f265000bd10 con 0x7f265c078ca0 2023-10-07T23:08:06.212 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.625+0000 7f265b7fe700 1 -- 172.21.15.67:0/2715450864 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2650011ce0 con 0x7f265c078ca0 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 -- 172.21.15.67:0/2715450864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 msgr2=0x7f265c0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 --2- 172.21.15.67:0/2715450864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c0771b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26500098e0 tx=0x7f2650009250 comp rx=0 tx=0).stop 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 -- 172.21.15.67:0/2715450864 shutdown_connections 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 --2- 172.21.15.67:0/2715450864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c0771b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 -- 172.21.15.67:0/2715450864 >> 172.21.15.67:0/2715450864 conn(0x7f265c0fda80 msgr2=0x7f265c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 -- 172.21.15.67:0/2715450864 shutdown_connections 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.626+0000 7f26631cc700 1 -- 172.21.15.67:0/2715450864 wait complete. 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.627+0000 7f26631cc700 1 Processor -- start 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.627+0000 7f26631cc700 1 -- start start 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.627+0000 7f26631cc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.627+0000 7f26631cc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c113780 con 0x7f265c078ca0 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.628+0000 7f2660f68700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:06.213 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.628+0000 7f2660f68700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53012/0 (socket says 172.21.15.67:53012) 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.628+0000 7f2660f68700 1 -- 172.21.15.67:0/1167085045 learned_addr learned my addr 172.21.15.67:0/1167085045 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.628+0000 7f2660f68700 1 -- 172.21.15.67:0/1167085045 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500094f0 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.628+0000 7f2660f68700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c113240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2650011970 tx=0x7f2650009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.629+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2650009e80 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.629+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c113980 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.629+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f265001aaf0 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.629+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2650022a90 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.629+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c113d90 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.630+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c04bb80 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.630+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f2650022400 con 0x7f265c078ca0 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.630+0000 7f26597fa700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f264c036610 0x7f264c038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:06.214 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.631+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f265004a4c0 con 0x7f265c078ca0 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.631+0000 7f265bfff700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f264c036610 0x7f264c038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.631+0000 7f265bfff700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f264c036610 0x7f264c038ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2648009990 tx=0x7f2648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.639+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f26500219f0 con 0x7f265c078ca0 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.844+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f265c0fef10 con 0x7f264c036610 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.845+0000 7f26597fa700 1 -- 172.21.15.67:0/1167085045 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f265c0fef10 con 0x7f264c036610 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f264c036610 msgr2=0x7f264c038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f264c036610 0x7f264c038ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2648009990 tx=0x7f2648006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 msgr2=0x7f265c113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c113240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2650011970 tx=0x7f2650009250 comp rx=0 tx=0).stop 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 shutdown_connections 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f264c036610 0x7f264c038ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:06.215 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 --2- 172.21.15.67:0/1167085045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c078ca0 0x7f265c113240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:06.216 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 >> 172.21.15.67:0/1167085045 conn(0x7f265c0fda80 msgr2=0x7f265c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:06.216 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 shutdown_connections 2023-10-07T23:08:06.216 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:05.847+0000 7f26631cc700 1 -- 172.21.15.67:0/1167085045 wait complete. 2023-10-07T23:08:06.216 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2023-10-07T23:08:07.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:06 smithi067 ceph-mon[82723]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:07.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:06 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:07.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:06 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:07 smithi067 ceph-mon[82723]: [07/Oct/2023:23:08:06] ENGINE Bus STARTING 2023-10-07T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:07 smithi067 ceph-mon[82723]: [07/Oct/2023:23:08:06] ENGINE Serving on https://172.21.15.67:7150 2023-10-07T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:07 smithi067 ceph-mon[82723]: [07/Oct/2023:23:08:06] ENGINE Bus STARTED 2023-10-07T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:07 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:07 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:07.956 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: Generating public/private rsa key pair. 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: Your identification has been saved in /tmp/tmp83ro822h/key. 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: Your public key has been saved in /tmp/tmp83ro822h/key.pub. 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: The key fingerprint is: 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: SHA256:bImM8xCZaNoJgTn+P3BIQ8aVke7Pz3lDhKvjquwUGpw ceph-227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: The key's randomart image is: 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: +---[RSA 3072]----+ 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: |o.. .o+ | 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: |+. = = | 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: |o.= = . | 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: |.*.+ = o o . | 2023-10-07T23:08:07.957 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: |.E=.B o S o | 2023-10-07T23:08:07.958 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: | o+.* . . . | 2023-10-07T23:08:07.958 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: | . .+ + . . | 2023-10-07T23:08:07.958 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: | o o =. .o | 2023-10-07T23:08:07.958 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: | .+..+.o+. . | 2023-10-07T23:08:07.958 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:07 smithi067 conmon[83139]: +----[SHA256]-----+ 2023-10-07T23:08:08.001 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.766+0000 7f8304084700 1 Processor -- start 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.766+0000 7f8304084700 1 -- start start 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.767+0000 7f8304084700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.767+0000 7f8304084700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc102c10 con 0x7f82fc1022c0 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.767+0000 7f8301e20700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.767+0000 7f8301e20700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53026/0 (socket says 172.21.15.67:53026) 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.767+0000 7f8301e20700 1 -- 172.21.15.67:0/904862567 learned_addr learned my addr 172.21.15.67:0/904862567 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.767+0000 7f8301e20700 1 -- 172.21.15.67:0/904862567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc102d50 con 0x7f82fc1022c0 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8301e20700 1 --2- 172.21.15.67:0/904862567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1026d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82f80098e0 tx=0x7f82f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4a05793834dc72 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:08.002 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8300e1e700 1 -- 172.21.15.67:0/904862567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82f8009e80 con 0x7f82fc1022c0 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8300e1e700 1 -- 172.21.15.67:0/904862567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82f800bd10 con 0x7f82fc1022c0 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8300e1e700 1 -- 172.21.15.67:0/904862567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82f8011ce0 con 0x7f82fc1022c0 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8304084700 1 -- 172.21.15.67:0/904862567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 msgr2=0x7f82fc1026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8304084700 1 --2- 172.21.15.67:0/904862567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1026d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82f80098e0 tx=0x7f82f8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8304084700 1 -- 172.21.15.67:0/904862567 shutdown_connections 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8304084700 1 --2- 172.21.15.67:0/904862567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1026d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.768+0000 7f8304084700 1 -- 172.21.15.67:0/904862567 >> 172.21.15.67:0/904862567 conn(0x7f82fc0fda40 msgr2=0x7f82fc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.769+0000 7f8304084700 1 -- 172.21.15.67:0/904862567 shutdown_connections 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.769+0000 7f8304084700 1 -- 172.21.15.67:0/904862567 wait complete. 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.769+0000 7f8304084700 1 Processor -- start 2023-10-07T23:08:08.003 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.769+0000 7f8304084700 1 -- start start 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.770+0000 7f8304084700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.770+0000 7f8304084700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc113710 con 0x7f82fc1022c0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.770+0000 7f8301e20700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.770+0000 7f8301e20700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53028/0 (socket says 172.21.15.67:53028) 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.770+0000 7f8301e20700 1 -- 172.21.15.67:0/1984718602 learned_addr learned my addr 172.21.15.67:0/1984718602 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.770+0000 7f8301e20700 1 -- 172.21.15.67:0/1984718602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80094f0 con 0x7f82fc1022c0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.771+0000 7f8301e20700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1131d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f82f8011970 tx=0x7f82f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.771+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82f801a670 con 0x7f82fc1022c0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.771+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82f801ac80 con 0x7f82fc1022c0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.771+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc113910 con 0x7f82fc1022c0 2023-10-07T23:08:08.004 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.771+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82f8022b30 con 0x7f82fc1022c0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.771+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc113d20 con 0x7f82fc1022c0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.772+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc04bb80 con 0x7f82fc1022c0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.772+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f82f8022d20 con 0x7f82fc1022c0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.772+0000 7f82f27fc700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f82e8036610 0x7f82e8038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.772+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f82f8049ab0 con 0x7f82fc1022c0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.773+0000 7f830161f700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f82e8036610 0x7f82e8038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.773+0000 7f830161f700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f82e8036610 0x7f82e8038ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009990 tx=0x7f82ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.777+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f82f80223c0 con 0x7f82fc1022c0 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:06.981+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f82fc0fee80 con 0x7f82e8036610 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.612+0000 7f82f27fc700 1 -- 172.21.15.67:0/1984718602 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f82fc0fee80 con 0x7f82e8036610 2023-10-07T23:08:08.005 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f82e8036610 msgr2=0x7f82e8038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f82e8036610 0x7f82e8038ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009990 tx=0x7f82ec006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 msgr2=0x7f82fc1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1131d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f82f8011970 tx=0x7f82f8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 shutdown_connections 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f82e8036610 0x7f82e8038ac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 --2- 172.21.15.67:0/1984718602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82fc1022c0 0x7f82fc1131d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 >> 172.21.15.67:0/1984718602 conn(0x7f82fc0fda40 msgr2=0x7f82fc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 shutdown_connections 2023-10-07T23:08:08.006 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:07.614+0000 7f8304084700 1 -- 172.21.15.67:0/1984718602 wait complete. 2023-10-07T23:08:09.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:08 smithi067 ceph-mon[82723]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:09.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:08 smithi067 ceph-mon[82723]: Generating ssh key... 2023-10-07T23:08:09.074 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.511+0000 7fbc59a02700 1 Processor -- start 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.511+0000 7fbc59a02700 1 -- start start 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.512+0000 7fbc59a02700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.512+0000 7fbc59a02700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54105180 con 0x7fbc541047a0 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.512+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.512+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53030/0 (socket says 172.21.15.67:53030) 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.512+0000 7fbc52ffd700 1 -- 172.21.15.67:0/3204006558 learned_addr learned my addr 172.21.15.67:0/3204006558 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.512+0000 7fbc52ffd700 1 -- 172.21.15.67:0/3204006558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc541052c0 con 0x7fbc541047a0 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.513+0000 7fbc52ffd700 1 --2- 172.21.15.67:0/3204006558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54104bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc440098e0 tx=0x7fbc44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bc40b3358809a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.513+0000 7fbc51ffb700 1 -- 172.21.15.67:0/3204006558 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc44009e80 con 0x7fbc541047a0 2023-10-07T23:08:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.513+0000 7fbc51ffb700 1 -- 172.21.15.67:0/3204006558 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc4400bd10 con 0x7fbc541047a0 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.513+0000 7fbc51ffb700 1 -- 172.21.15.67:0/3204006558 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc44011ce0 con 0x7fbc541047a0 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 -- 172.21.15.67:0/3204006558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 msgr2=0x7fbc54104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 --2- 172.21.15.67:0/3204006558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54104bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc440098e0 tx=0x7fbc44009250 comp rx=0 tx=0).stop 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 -- 172.21.15.67:0/3204006558 shutdown_connections 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 --2- 172.21.15.67:0/3204006558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54104bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 -- 172.21.15.67:0/3204006558 >> 172.21.15.67:0/3204006558 conn(0x7fbc54077ab0 msgr2=0x7fbc54077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 -- 172.21.15.67:0/3204006558 shutdown_connections 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.514+0000 7fbc59a02700 1 -- 172.21.15.67:0/3204006558 wait complete. 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc59a02700 1 Processor -- start 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc59a02700 1 -- start start 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc59a02700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:09.076 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc59a02700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc541136b0 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53040/0 (socket says 172.21.15.67:53040) 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.515+0000 7fbc52ffd700 1 -- 172.21.15.67:0/1564812996 learned_addr learned my addr 172.21.15.67:0/1564812996 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc52ffd700 1 -- 172.21.15.67:0/1564812996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440094f0 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc52ffd700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54113170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006b00 tx=0x7fbc44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc4401a640 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc54113910 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc4401ac50 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc44022bb0 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.516+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc54114550 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.517+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7fbc44022490 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.517+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5405f270 con 0x7fbc541047a0 2023-10-07T23:08:09.077 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.518+0000 7fbc3b7fe700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fbc3c0365b0 0x7fbc3c038a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.518+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbc4404a420 con 0x7fbc541047a0 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.518+0000 7fbc527fc700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fbc3c0365b0 0x7fbc3c038a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.519+0000 7fbc527fc700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fbc3c0365b0 0x7fbc3c038a60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009990 tx=0x7fbc48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.523+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fbc440219f0 con 0x7fbc541047a0 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.729+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc54102d00 con 0x7fbc3c0365b0 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.730+0000 7fbc3b7fe700 1 -- 172.21.15.67:0/1564812996 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fbc54102d00 con 0x7fbc3c0365b0 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fbc3c0365b0 msgr2=0x7fbc3c038a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fbc3c0365b0 0x7fbc3c038a60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009990 tx=0x7fbc48006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 msgr2=0x7fbc54113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54113170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006b00 tx=0x7fbc44009250 comp rx=0 tx=0).stop 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 shutdown_connections 2023-10-07T23:08:09.078 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fbc3c0365b0 0x7fbc3c038a60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 --2- 172.21.15.67:0/1564812996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc541047a0 0x7fbc54113170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 >> 172.21.15.67:0/1564812996 conn(0x7fbc54077ab0 msgr2=0x7fbc541025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 shutdown_connections 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:08.733+0000 7fbc59a02700 1 -- 172.21.15.67:0/1564812996 wait complete. 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2023-10-07T23:08:09.079 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2023-10-07T23:08:09.973 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:09 smithi067 ceph-mon[82723]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:10.891 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:10 smithi067 ceph-mon[82723]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:11.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:11 smithi067 ceph-mon[82723]: Deploying cephadm binary to smithi067 2023-10-07T23:08:12.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2023-10-07T23:08:12.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.600+0000 7f01cb1eb700 1 Processor -- start 2023-10-07T23:08:12.757 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.600+0000 7f01cb1eb700 1 -- start start 2023-10-07T23:08:12.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.601+0000 7f01cb1eb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:12.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.601+0000 7f01cb1eb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4079130 con 0x7f01c4078d20 2023-10-07T23:08:12.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.601+0000 7f01c8f87700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:12.758 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.601+0000 7f01c8f87700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53046/0 (socket says 172.21.15.67:53046) 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.601+0000 7f01c8f87700 1 -- 172.21.15.67:0/1828411702 learned_addr learned my addr 172.21.15.67:0/1828411702 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.601+0000 7f01c8f87700 1 -- 172.21.15.67:0/1828411702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4077770 con 0x7f01c4078d20 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.602+0000 7f01c8f87700 1 --2- 172.21.15.67:0/1828411702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4077230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01b80098e0 tx=0x7f01b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c613d6ff68c7123 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.602+0000 7f01c37fe700 1 -- 172.21.15.67:0/1828411702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b8009e80 con 0x7f01c4078d20 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.602+0000 7f01c37fe700 1 -- 172.21.15.67:0/1828411702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01b800bd10 con 0x7f01c4078d20 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.602+0000 7f01c37fe700 1 -- 172.21.15.67:0/1828411702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b8011e20 con 0x7f01c4078d20 2023-10-07T23:08:12.759 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 -- 172.21.15.67:0/1828411702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 msgr2=0x7f01c4077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 --2- 172.21.15.67:0/1828411702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4077230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01b80098e0 tx=0x7f01b8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 -- 172.21.15.67:0/1828411702 shutdown_connections 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 --2- 172.21.15.67:0/1828411702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4077230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 -- 172.21.15.67:0/1828411702 >> 172.21.15.67:0/1828411702 conn(0x7f01c40fda80 msgr2=0x7f01c40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 -- 172.21.15.67:0/1828411702 shutdown_connections 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.603+0000 7f01cb1eb700 1 -- 172.21.15.67:0/1828411702 wait complete. 2023-10-07T23:08:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01cb1eb700 1 Processor -- start 2023-10-07T23:08:12.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01cb1eb700 1 -- start start 2023-10-07T23:08:12.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01cb1eb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4113380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:12.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01cb1eb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c41138c0 con 0x7f01c4078d20 2023-10-07T23:08:12.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01c8f87700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4113380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:12.761 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01c8f87700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4113380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53060/0 (socket says 172.21.15.67:53060) 2023-10-07T23:08:12.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.604+0000 7f01c8f87700 1 -- 172.21.15.67:0/895183872 learned_addr learned my addr 172.21.15.67:0/895183872 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:12.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01c8f87700 1 -- 172.21.15.67:0/895183872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80094f0 con 0x7f01c4078d20 2023-10-07T23:08:12.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01c8f87700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4113380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006fd0 tx=0x7f01b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:12.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b80117a0 con 0x7f01c4078d20 2023-10-07T23:08:12.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c4113ac0 con 0x7f01c4078d20 2023-10-07T23:08:12.762 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4113ed0 con 0x7f01c4078d20 2023-10-07T23:08:12.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01b8009e80 con 0x7f01c4078d20 2023-10-07T23:08:12.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.605+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b801ab60 con 0x7f01c4078d20 2023-10-07T23:08:12.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.606+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f01b801acc0 con 0x7f01c4078d20 2023-10-07T23:08:12.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.606+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a80052e0 con 0x7f01c4078d20 2023-10-07T23:08:12.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.607+0000 7f01c17fa700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f01b40365b0 0x7f01b4038a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:12.763 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.607+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f01b804a930 con 0x7f01c4078d20 2023-10-07T23:08:12.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.607+0000 7f01c3fff700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f01b40365b0 0x7f01b4038a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:12.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.608+0000 7f01c3fff700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f01b40365b0 0x7f01b4038a60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009990 tx=0x7f01b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:12.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.612+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f01b8020940 con 0x7f01c4078d20 2023-10-07T23:08:12.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:09.817+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}) v1 -- 0x7f01a8000bf0 con 0x7f01b40365b0 2023-10-07T23:08:12.764 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.462+0000 7f01c17fa700 1 -- 172.21.15.67:0/895183872 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f01a8000bf0 con 0x7f01b40365b0 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f01b40365b0 msgr2=0x7f01b4038a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f01b40365b0 0x7f01b4038a60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009990 tx=0x7f01b0006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 msgr2=0x7f01c4113380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4113380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006fd0 tx=0x7f01b8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 shutdown_connections 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f01b40365b0 0x7f01b4038a60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 --2- 172.21.15.67:0/895183872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4078d20 0x7f01c4113380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.465+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 >> 172.21.15.67:0/895183872 conn(0x7f01c40fda80 msgr2=0x7f01c40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.466+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 shutdown_connections 2023-10-07T23:08:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:12.466+0000 7f01cb1eb700 1 -- 172.21.15.67:0/895183872 wait complete. 2023-10-07T23:08:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2023-10-07T23:08:13.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:13 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:13.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:13 smithi067 ceph-mon[82723]: Added host smithi067 2023-10-07T23:08:13.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:13 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:13.988 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-07T23:08:13.988 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.272+0000 7fc4ce584700 1 Processor -- start 2023-10-07T23:08:13.988 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.272+0000 7fc4ce584700 1 -- start start 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.272+0000 7fc4ce584700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.273+0000 7fc4ce584700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8102c70 con 0x7fc4c8102320 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.273+0000 7fc4c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.273+0000 7fc4c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44208/0 (socket says 172.21.15.67:44208) 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.273+0000 7fc4c7fff700 1 -- 172.21.15.67:0/77267059 learned_addr learned my addr 172.21.15.67:0/77267059 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.273+0000 7fc4c7fff700 1 -- 172.21.15.67:0/77267059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8102db0 con 0x7fc4c8102320 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.274+0000 7fc4c7fff700 1 --2- 172.21.15.67:0/77267059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8102730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098e0 tx=0x7fc4b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d380eb161f40c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.274+0000 7fc4c6ffd700 1 -- 172.21.15.67:0/77267059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8009e80 con 0x7fc4c8102320 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.274+0000 7fc4c6ffd700 1 -- 172.21.15.67:0/77267059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4b800bd10 con 0x7fc4c8102320 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.274+0000 7fc4c6ffd700 1 -- 172.21.15.67:0/77267059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8011ce0 con 0x7fc4c8102320 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.274+0000 7fc4ce584700 1 -- 172.21.15.67:0/77267059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 msgr2=0x7fc4c8102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:13.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.274+0000 7fc4ce584700 1 --2- 172.21.15.67:0/77267059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8102730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098e0 tx=0x7fc4b8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.275+0000 7fc4ce584700 1 -- 172.21.15.67:0/77267059 shutdown_connections 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.275+0000 7fc4ce584700 1 --2- 172.21.15.67:0/77267059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8102730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.275+0000 7fc4ce584700 1 -- 172.21.15.67:0/77267059 >> 172.21.15.67:0/77267059 conn(0x7fc4c80fda80 msgr2=0x7fc4c80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.275+0000 7fc4ce584700 1 -- 172.21.15.67:0/77267059 shutdown_connections 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.275+0000 7fc4ce584700 1 -- 172.21.15.67:0/77267059 wait complete. 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.275+0000 7fc4ce584700 1 Processor -- start 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4ce584700 1 -- start start 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4ce584700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4ce584700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8077190 con 0x7fc4c8102320 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44214/0 (socket says 172.21.15.67:44214) 2023-10-07T23:08:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4c7fff700 1 -- 172.21.15.67:0/403487383 learned_addr learned my addr 172.21.15.67:0/403487383 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.276+0000 7fc4c7fff700 1 -- 172.21.15.67:0/403487383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b80094f0 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.277+0000 7fc4c7fff700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8078d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8006b00 tx=0x7fc4b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.277+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4b801a640 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.277+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4b801ac50 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.277+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8022bb0 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.277+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c8077390 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.277+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c80777a0 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.278+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7fc4b8022490 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.278+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c804bb80 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.279+0000 7fc4c4ff9700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc4b0036610 0x7fc4b0038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.279+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc4b804a4a0 con 0x7fc4c8102320 2023-10-07T23:08:13.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.279+0000 7fc4c77fe700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc4b0036610 0x7fc4b0038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.279+0000 7fc4c77fe700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc4b0036610 0x7fc4b0038ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009990 tx=0x7fc4bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.283+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fc4b801adc0 con 0x7fc4c8102320 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.499+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c8077fe0 con 0x7fc4b0036610 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.504+0000 7fc4c4ff9700 1 -- 172.21.15.67:0/403487383 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc4c8077fe0 con 0x7fc4b0036610 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc4b0036610 msgr2=0x7fc4b0038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc4b0036610 0x7fc4b0038ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009990 tx=0x7fc4bc006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 msgr2=0x7fc4c8078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8078d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8006b00 tx=0x7fc4b8009250 comp rx=0 tx=0).stop 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 shutdown_connections 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fc4b0036610 0x7fc4b0038ac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 --2- 172.21.15.67:0/403487383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4c8102320 0x7fc4c8078d10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:13.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.506+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 >> 172.21.15.67:0/403487383 conn(0x7fc4c80fda80 msgr2=0x7fc4c80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:13.993 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.507+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 shutdown_connections 2023-10-07T23:08:13.993 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:13.507+0000 7fc4ce584700 1 -- 172.21.15.67:0/403487383 wait complete. 2023-10-07T23:08:13.993 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2023-10-07T23:08:14.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:14 smithi067 ceph-mon[82723]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:14.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:14 smithi067 ceph-mon[82723]: Saving service mon spec with placement count:5 2023-10-07T23:08:14.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:14 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.533+0000 7f16336ca700 1 Processor -- start 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.533+0000 7f16336ca700 1 -- start start 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.533+0000 7f16336ca700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.533+0000 7f16336ca700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c079110 con 0x7f162c078d00 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.534+0000 7f1631466700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.534+0000 7f1631466700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44218/0 (socket says 172.21.15.67:44218) 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.534+0000 7f1631466700 1 -- 172.21.15.67:0/3365951836 learned_addr learned my addr 172.21.15.67:0/3365951836 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.534+0000 7f1631466700 1 -- 172.21.15.67:0/3365951836 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c077750 con 0x7f162c078d00 2023-10-07T23:08:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.534+0000 7f1631466700 1 --2- 172.21.15.67:0/3365951836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c077210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16280098e0 tx=0x7f1628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d2369da0b929b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f1623fff700 1 -- 172.21.15.67:0/3365951836 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1628009e80 con 0x7f162c078d00 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f1623fff700 1 -- 172.21.15.67:0/3365951836 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f162800bd10 con 0x7f162c078d00 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f1623fff700 1 -- 172.21.15.67:0/3365951836 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1628011ce0 con 0x7f162c078d00 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 -- 172.21.15.67:0/3365951836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 msgr2=0x7f162c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 --2- 172.21.15.67:0/3365951836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c077210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16280098e0 tx=0x7f1628009250 comp rx=0 tx=0).stop 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 -- 172.21.15.67:0/3365951836 shutdown_connections 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 --2- 172.21.15.67:0/3365951836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c077210 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 -- 172.21.15.67:0/3365951836 >> 172.21.15.67:0/3365951836 conn(0x7f162c0fda80 msgr2=0x7f162c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 -- 172.21.15.67:0/3365951836 shutdown_connections 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.535+0000 7f16336ca700 1 -- 172.21.15.67:0/3365951836 wait complete. 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.536+0000 7f16336ca700 1 Processor -- start 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.536+0000 7f16336ca700 1 -- start start 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f16336ca700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f16336ca700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c113770 con 0x7f162c078d00 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f1631466700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f1631466700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44220/0 (socket says 172.21.15.67:44220) 2023-10-07T23:08:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f1631466700 1 -- 172.21.15.67:0/2099574202 learned_addr learned my addr 172.21.15.67:0/2099574202 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f1631466700 1 -- 172.21.15.67:0/2099574202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280094f0 con 0x7f162c078d00 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.537+0000 7f1631466700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c113230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1628011970 tx=0x7f1628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.538+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1628009e80 con 0x7f162c078d00 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.538+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f162801aaf0 con 0x7f162c078d00 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.538+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1628022a90 con 0x7f162c078d00 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.538+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c113970 con 0x7f162c078d00 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.538+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c113d80 con 0x7f162c078d00 2023-10-07T23:08:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.539+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c04bb80 con 0x7f162c078d00 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.539+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f1628022400 con 0x7f162c078d00 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.539+0000 7f1621ffb700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1618036610 0x7f1618038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.539+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f162804a500 con 0x7f162c078d00 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.540+0000 7f1630c65700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1618036610 0x7f1618038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.540+0000 7f1630c65700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1618036610 0x7f1618038ac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f161c009990 tx=0x7f161c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.544+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f16280219f0 con 0x7f162c078d00 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.765+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f162c0feed0 con 0x7f1618036610 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.769+0000 7f1621ffb700 1 -- 172.21.15.67:0/2099574202 <== mgr.14120 v2:172.21.15.67:6800/2928618291 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f162c0feed0 con 0x7f1618036610 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1618036610 msgr2=0x7f1618038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1618036610 0x7f1618038ac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f161c009990 tx=0x7f161c006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 msgr2=0x7f162c113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c113230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1628011970 tx=0x7f1628009250 comp rx=0 tx=0).stop 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 shutdown_connections 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f1618036610 0x7f1618038ac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 --2- 172.21.15.67:0/2099574202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f162c078d00 0x7f162c113230 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 >> 172.21.15.67:0/2099574202 conn(0x7f162c0fda80 msgr2=0x7f162c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 shutdown_connections 2023-10-07T23:08:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:14.772+0000 7f16336ca700 1 -- 172.21.15.67:0/2099574202 wait complete. 2023-10-07T23:08:16.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:15 smithi067 ceph-mon[82723]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:16.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:15 smithi067 ceph-mon[82723]: Saving service mgr spec with placement count:2 2023-10-07T23:08:16.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:15 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:16.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:15 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:16.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:15 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:16.451 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.697+0000 7f0bf8272700 1 Processor -- start 2023-10-07T23:08:16.451 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.697+0000 7f0bf8272700 1 -- start start 2023-10-07T23:08:16.451 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.697+0000 7f0bf8272700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:16.451 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.697+0000 7f0bf8272700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0102c70 con 0x7f0bf0102320 2023-10-07T23:08:16.451 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.698+0000 7f0bf600e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:16.451 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.698+0000 7f0bf600e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44228/0 (socket says 172.21.15.67:44228) 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.698+0000 7f0bf600e700 1 -- 172.21.15.67:0/1595968458 learned_addr learned my addr 172.21.15.67:0/1595968458 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.698+0000 7f0bf600e700 1 -- 172.21.15.67:0/1595968458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0102db0 con 0x7f0bf0102320 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.699+0000 7f0bf600e700 1 --2- 172.21.15.67:0/1595968458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0102730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0098e0 tx=0x7f0bec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d2d7b2f55399c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.699+0000 7f0bf500c700 1 -- 172.21.15.67:0/1595968458 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec009e80 con 0x7f0bf0102320 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.699+0000 7f0bf500c700 1 -- 172.21.15.67:0/1595968458 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bec00bd30 con 0x7f0bf0102320 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.699+0000 7f0bf500c700 1 -- 172.21.15.67:0/1595968458 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec011ca0 con 0x7f0bf0102320 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.699+0000 7f0bf8272700 1 -- 172.21.15.67:0/1595968458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 msgr2=0x7f0bf0102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.699+0000 7f0bf8272700 1 --2- 172.21.15.67:0/1595968458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0102730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0098e0 tx=0x7f0bec009250 comp rx=0 tx=0).stop 2023-10-07T23:08:16.452 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.700+0000 7f0bf8272700 1 -- 172.21.15.67:0/1595968458 shutdown_connections 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.700+0000 7f0bf8272700 1 --2- 172.21.15.67:0/1595968458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0102730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.700+0000 7f0bf8272700 1 -- 172.21.15.67:0/1595968458 >> 172.21.15.67:0/1595968458 conn(0x7f0bf00fda80 msgr2=0x7f0bf00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.700+0000 7f0bf8272700 1 -- 172.21.15.67:0/1595968458 shutdown_connections 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.700+0000 7f0bf8272700 1 -- 172.21.15.67:0/1595968458 wait complete. 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.700+0000 7f0bf8272700 1 Processor -- start 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.701+0000 7f0bf8272700 1 -- start start 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.701+0000 7f0bf8272700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0111030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.701+0000 7f0bf8272700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0111570 con 0x7f0bf0102320 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.701+0000 7f0bf600e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0111030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:16.453 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.701+0000 7f0bf600e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0111030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44244/0 (socket says 172.21.15.67:44244) 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.701+0000 7f0bf600e700 1 -- 172.21.15.67:0/782000019 learned_addr learned my addr 172.21.15.67:0/782000019 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0bf600e700 1 -- 172.21.15.67:0/782000019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0094f0 con 0x7f0bf0102320 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0bf600e700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0111030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec01a760 con 0x7f0bf0102320 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bec01ad90 con 0x7f0bf0102320 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec022c60 con 0x7f0bf0102320 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf0111770 con 0x7f0bf0102320 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.702+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf0111ac0 con 0x7f0bf0102320 2023-10-07T23:08:16.454 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.703+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f0bec022570 con 0x7f0bf0102320 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.704+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf004bb80 con 0x7f0bf0102320 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.704+0000 7f0be6ffd700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f0bdc036610 0x7f0bdc038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.704+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0bec04a690 con 0x7f0bf0102320 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.704+0000 7f0bf580d700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f0bdc036610 0x7f0bdc038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.705+0000 7f0bf580d700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f0bdc036610 0x7f0bdc038ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009990 tx=0x7f0be0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.709+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f0bec02ab80 con 0x7f0bf0102320 2023-10-07T23:08:16.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.913+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0bf0104220 con 0x7f0bf0102320 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.926+0000 7f0be6ffd700 1 -- 172.21.15.67:0/782000019 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0bec02ab80 con 0x7f0bf0102320 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.928+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f0bdc036610 msgr2=0x7f0bdc038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.928+0000 7f0bf8272700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f0bdc036610 0x7f0bdc038ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009990 tx=0x7f0be0006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.928+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 msgr2=0x7f0bf0111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.928+0000 7f0bf8272700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0111030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec009250 comp rx=0 tx=0).stop 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.929+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 shutdown_connections 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.929+0000 7f0bf8272700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f0bdc036610 0x7f0bdc038ac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.929+0000 7f0bf8272700 1 --2- 172.21.15.67:0/782000019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bf0102320 0x7f0bf0111030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:16.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.929+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 >> 172.21.15.67:0/782000019 conn(0x7f0bf00fda80 msgr2=0x7f0bf00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:16.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.929+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 shutdown_connections 2023-10-07T23:08:16.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:15.929+0000 7f0bf8272700 1 -- 172.21.15.67:0/782000019 wait complete. 2023-10-07T23:08:17.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:16 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/782000019' entity='client.admin' 2023-10-07T23:08:17.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.003+0000 7f9bb17a5700 1 Processor -- start 2023-10-07T23:08:17.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.003+0000 7f9bb17a5700 1 -- start start 2023-10-07T23:08:17.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.003+0000 7f9bb17a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac0771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:17.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.003+0000 7f9bb17a5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0790f0 con 0x7f9bac078ce0 2023-10-07T23:08:17.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.004+0000 7f9baaffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac0771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:17.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.004+0000 7f9baaffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac0771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44252/0 (socket says 172.21.15.67:44252) 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.004+0000 7f9baaffd700 1 -- 172.21.15.67:0/3623825425 learned_addr learned my addr 172.21.15.67:0/3623825425 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.004+0000 7f9baaffd700 1 -- 172.21.15.67:0/3623825425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac077730 con 0x7f9bac078ce0 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.004+0000 7f9baaffd700 1 --2- 172.21.15.67:0/3623825425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac0771f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0098e0 tx=0x7f9b9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d5e9ac266beb23 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9ba9ffb700 1 -- 172.21.15.67:0/3623825425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b9c009e80 con 0x7f9bac078ce0 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9ba9ffb700 1 -- 172.21.15.67:0/3623825425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9c00bd30 con 0x7f9bac078ce0 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9ba9ffb700 1 -- 172.21.15.67:0/3623825425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b9c011ca0 con 0x7f9bac078ce0 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 -- 172.21.15.67:0/3623825425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 msgr2=0x7f9bac0771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:17.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 --2- 172.21.15.67:0/3623825425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac0771f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0098e0 tx=0x7f9b9c009250 comp rx=0 tx=0).stop 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 -- 172.21.15.67:0/3623825425 shutdown_connections 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 --2- 172.21.15.67:0/3623825425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac0771f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 -- 172.21.15.67:0/3623825425 >> 172.21.15.67:0/3623825425 conn(0x7f9bac0fda80 msgr2=0x7f9bac0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 -- 172.21.15.67:0/3623825425 shutdown_connections 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.005+0000 7f9bb17a5700 1 -- 172.21.15.67:0/3623825425 wait complete. 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.006+0000 7f9bb17a5700 1 Processor -- start 2023-10-07T23:08:17.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.006+0000 7f9bb17a5700 1 -- start start 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9bb17a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9bb17a5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac113760 con 0x7f9bac078ce0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9baaffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9baaffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44262/0 (socket says 172.21.15.67:44262) 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9baaffd700 1 -- 172.21.15.67:0/1698306615 learned_addr learned my addr 172.21.15.67:0/1698306615 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9baaffd700 1 -- 172.21.15.67:0/1698306615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0094f0 con 0x7f9bac078ce0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.007+0000 7f9baaffd700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac113220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006fd0 tx=0x7f9b9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.008+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b9c01a710 con 0x7f9bac078ce0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.008+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9c01ad40 con 0x7f9bac078ce0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.008+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac113960 con 0x7f9bac078ce0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.008+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b9c022c90 con 0x7f9bac078ce0 2023-10-07T23:08:17.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.008+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac113d70 con 0x7f9bac078ce0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.009+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7f9b9c022500 con 0x7f9bac078ce0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.009+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac04bb80 con 0x7f9bac078ce0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.009+0000 7f9b937fe700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f9b94036610 0x7f9b94038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.009+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9b9c04a6d0 con 0x7f9bac078ce0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.010+0000 7f9baa7fc700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f9b94036610 0x7f9b94038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.010+0000 7f9baa7fc700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f9b94036610 0x7f9b94038ac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009990 tx=0x7f9ba0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.014+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9b9c021bd0 con 0x7f9bac078ce0 2023-10-07T23:08:17.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.219+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f9bac1086b0 con 0x7f9bac078ce0 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.223+0000 7f9b937fe700 1 -- 172.21.15.67:0/1698306615 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f9b9c0213a0 con 0x7f9bac078ce0 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.226+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f9b94036610 msgr2=0x7f9b94038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.226+0000 7f9bb17a5700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f9b94036610 0x7f9b94038ac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0009990 tx=0x7f9ba0006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.226+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 msgr2=0x7f9bac113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.226+0000 7f9bb17a5700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac113220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006fd0 tx=0x7f9b9c009250 comp rx=0 tx=0).stop 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.227+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 shutdown_connections 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.227+0000 7f9bb17a5700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f9b94036610 0x7f9b94038ac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.227+0000 7f9bb17a5700 1 --2- 172.21.15.67:0/1698306615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bac078ce0 0x7f9bac113220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:17.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.227+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 >> 172.21.15.67:0/1698306615 conn(0x7f9bac0fda80 msgr2=0x7f9bac0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:17.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.227+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 shutdown_connections 2023-10-07T23:08:17.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:17.227+0000 7f9bb17a5700 1 -- 172.21.15.67:0/1698306615 wait complete. 2023-10-07T23:08:17.733 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2023-10-07T23:08:18.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:18 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1698306615' entity='client.admin' 2023-10-07T23:08:18.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:18 smithi067 ceph-mon[82723]: from='mgr.14120 172.21.15.67:0/329734236' entity='mgr.x' 2023-10-07T23:08:19.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:19 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/255746620' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-07T23:08:19.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:19 smithi067 conmon[83139]: ignoring --setuser 2023-10-07T23:08:19.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:19 smithi067 conmon[83139]: ceph since I am not root 2023-10-07T23:08:19.505 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:19 smithi067 conmon[83139]: ignoring --setgroup ceph since I am not root 2023-10-07T23:08:19.607 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.252+0000 7fcbd5f39700 1 Processor -- start 2023-10-07T23:08:19.607 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.252+0000 7fcbd5f39700 1 -- start start 2023-10-07T23:08:19.607 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbd5f39700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:19.607 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbd5f39700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0102c70 con 0x7fcbd0102320 2023-10-07T23:08:19.607 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbcf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbcf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44270/0 (socket says 172.21.15.67:44270) 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbcf7fe700 1 -- 172.21.15.67:0/2647647 learned_addr learned my addr 172.21.15.67:0/2647647 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbcf7fe700 1 -- 172.21.15.67:0/2647647 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0102db0 con 0x7fcbd0102320 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.253+0000 7fcbcf7fe700 1 --2- 172.21.15.67:0/2647647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0102730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00098e0 tx=0x7fcbc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b3b50883965c08 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.254+0000 7fcbce7fc700 1 -- 172.21.15.67:0/2647647 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc0009e80 con 0x7fcbd0102320 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.254+0000 7fcbce7fc700 1 -- 172.21.15.67:0/2647647 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbc000bd30 con 0x7fcbd0102320 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.254+0000 7fcbce7fc700 1 -- 172.21.15.67:0/2647647 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc0011ca0 con 0x7fcbd0102320 2023-10-07T23:08:19.608 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.254+0000 7fcbd5f39700 1 -- 172.21.15.67:0/2647647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 msgr2=0x7fcbd0102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.254+0000 7fcbd5f39700 1 --2- 172.21.15.67:0/2647647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0102730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00098e0 tx=0x7fcbc0009250 comp rx=0 tx=0).stop 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.254+0000 7fcbd5f39700 1 -- 172.21.15.67:0/2647647 shutdown_connections 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.255+0000 7fcbd5f39700 1 --2- 172.21.15.67:0/2647647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0102730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.255+0000 7fcbd5f39700 1 -- 172.21.15.67:0/2647647 >> 172.21.15.67:0/2647647 conn(0x7fcbd00fda80 msgr2=0x7fcbd00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.255+0000 7fcbd5f39700 1 -- 172.21.15.67:0/2647647 shutdown_connections 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.255+0000 7fcbd5f39700 1 -- 172.21.15.67:0/2647647 wait complete. 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.256+0000 7fcbd5f39700 1 Processor -- start 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.256+0000 7fcbd5f39700 1 -- start start 2023-10-07T23:08:19.609 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.256+0000 7fcbd5f39700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.256+0000 7fcbd5f39700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0111520 con 0x7fcbd0102320 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.256+0000 7fcbcf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbcf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44276/0 (socket says 172.21.15.67:44276) 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbcf7fe700 1 -- 172.21.15.67:0/255746620 learned_addr learned my addr 172.21.15.67:0/255746620 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbcf7fe700 1 -- 172.21.15.67:0/255746620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00094f0 con 0x7fcbd0102320 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbcf7fe700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0110fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0000c00 tx=0x7fcbc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc001a760 con 0x7fcbd0102320 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd0111720 con 0x7fcbd0102320 2023-10-07T23:08:19.610 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbc001ad90 con 0x7fcbd0102320 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.257+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc0022c60 con 0x7fcbd0102320 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.258+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd0111b30 con 0x7fcbd0102320 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.259+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd004bb80 con 0x7fcbd0102320 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.259+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 43265+0+0 (secure 0 0 0) 0x7fcbc0022570 con 0x7fcbd0102320 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.259+0000 7fcbb7fff700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fcbb80365b0 0x7fcbb8038a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.259+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fcbc004a670 con 0x7fcbd0102320 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.260+0000 7fcbceffd700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fcbb80365b0 0x7fcbb8038a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:19.611 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.260+0000 7fcbceffd700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fcbb80365b0 0x7fcbb8038a60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009990 tx=0x7fcbc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.264+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fcbc002ad60 con 0x7fcbd0102320 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:18.534+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fcbd0104220 con 0x7fcbd0102320 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.232+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 9) v1 ==== 43278+0+0 (secure 0 0 0) 0x7fcbc002c960 con 0x7fcbd0102320 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.234+0000 7fcbb7fff700 1 -- 172.21.15.67:0/255746620 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fcbc004ad10 con 0x7fcbd0102320 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fcbb80365b0 msgr2=0x7fcbb8038a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fcbb80365b0 0x7fcbb8038a60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009990 tx=0x7fcbc4006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 msgr2=0x7fcbd0110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0110fe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0000c00 tx=0x7fcbc0009250 comp rx=0 tx=0).stop 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 shutdown_connections 2023-10-07T23:08:19.612 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7fcbb80365b0 0x7fcbb8038a60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:19.613 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 --2- 172.21.15.67:0/255746620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd0102320 0x7fcbd0110fe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:19.613 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 >> 172.21.15.67:0/255746620 conn(0x7fcbd00fda80 msgr2=0x7fcbd00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:19.613 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 shutdown_connections 2023-10-07T23:08:19.613 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:19.237+0000 7fcbd5f39700 1 -- 172.21.15.67:0/255746620 wait complete. 2023-10-07T23:08:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:20 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/255746620' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-07T23:08:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:20 smithi067 ceph-mon[82723]: mgrmap e9: x(active, since 16s) 2023-10-07T23:08:20.878 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.132+0000 7f6085b30700 1 Processor -- start 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.132+0000 7f6085b30700 1 -- start start 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.133+0000 7f6085b30700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f60800771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:20.884 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.133+0000 7f6085b30700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800790f0 con 0x7f6080078ce0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.133+0000 7f607f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f60800771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.133+0000 7f607f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f60800771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44314/0 (socket says 172.21.15.67:44314) 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.133+0000 7f607f7fe700 1 -- 172.21.15.67:0/1815899651 learned_addr learned my addr 172.21.15.67:0/1815899651 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.134+0000 7f607f7fe700 1 -- 172.21.15.67:0/1815899651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080077730 con 0x7f6080078ce0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.134+0000 7f607f7fe700 1 --2- 172.21.15.67:0/1815899651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f60800771f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60700098e0 tx=0x7f6070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfaaa93fbc1c861 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.134+0000 7f607e7fc700 1 -- 172.21.15.67:0/1815899651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6070009e80 con 0x7f6080078ce0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.134+0000 7f607e7fc700 1 -- 172.21.15.67:0/1815899651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607000bd30 con 0x7f6080078ce0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.134+0000 7f607e7fc700 1 -- 172.21.15.67:0/1815899651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6070011ca0 con 0x7f6080078ce0 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 -- 172.21.15.67:0/1815899651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 msgr2=0x7f60800771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:20.885 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 --2- 172.21.15.67:0/1815899651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f60800771f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60700098e0 tx=0x7f6070009250 comp rx=0 tx=0).stop 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 -- 172.21.15.67:0/1815899651 shutdown_connections 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 --2- 172.21.15.67:0/1815899651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f60800771f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 -- 172.21.15.67:0/1815899651 >> 172.21.15.67:0/1815899651 conn(0x7f60800fda80 msgr2=0x7f60800ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 -- 172.21.15.67:0/1815899651 shutdown_connections 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.135+0000 7f6085b30700 1 -- 172.21.15.67:0/1815899651 wait complete. 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f6085b30700 1 Processor -- start 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f6085b30700 1 -- start start 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f6085b30700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f6080113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f6085b30700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60801137a0 con 0x7f6080078ce0 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f607f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f6080113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f607f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f6080113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44324/0 (socket says 172.21.15.67:44324) 2023-10-07T23:08:20.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.136+0000 7f607f7fe700 1 -- 172.21.15.67:0/765562359 learned_addr learned my addr 172.21.15.67:0/765562359 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.137+0000 7f607f7fe700 1 -- 172.21.15.67:0/765562359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700094f0 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.137+0000 7f607f7fe700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f6080113260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6070006fd0 tx=0x7f6070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.137+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f607001a710 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.137+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607001ad40 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.137+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60801139a0 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.137+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6070022c90 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.138+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6080113db0 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.138+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) v1 ==== 43278+0+0 (secure 0 0 0) 0x7f6070022500 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.139+0000 7f6067fff700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 0x7f6068038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.139+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f607004a720 con 0x7f6080078ce0 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.139+0000 7f607effd700 1 -- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 msgr2=0x7f6068038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:20.887 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.139+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608004bb80 con 0x7f6080078ce0 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.139+0000 7f607effd700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 0x7f6068038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.145+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f6070021e00 con 0x7f6080078ce0 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.340+0000 7f607effd700 1 -- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 msgr2=0x7f6068038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.340+0000 7f607effd700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 0x7f6068038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.425+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6080114a80 con 0x7f6080078ce0 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.425+0000 7f6067fff700 1 -- 172.21.15.67:0/765562359 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6070048070 con 0x7f6080078ce0 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 msgr2=0x7f6068038ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 0x7f6068038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 msgr2=0x7f6080113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f6080113260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6070006fd0 tx=0x7f6070009250 comp rx=0 tx=0).stop 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 shutdown_connections 2023-10-07T23:08:20.888 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f6068036610 0x7f6068038ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:20.889 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 --2- 172.21.15.67:0/765562359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6080078ce0 0x7f6080113260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:20.889 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 >> 172.21.15.67:0/765562359 conn(0x7f60800fda80 msgr2=0x7f60800fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:20.889 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 shutdown_connections 2023-10-07T23:08:20.889 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:20.428+0000 7f6085b30700 1 -- 172.21.15.67:0/765562359 wait complete. 2023-10-07T23:08:20.889 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2023-10-07T23:08:20.889 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2023-10-07T23:08:21.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:21 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/765562359' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-07T23:08:21.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:20 smithi067 conmon[83139]: 2023-10-07T23:08:20.988+0000 7f85551261c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:08:21.255 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:21 smithi067 conmon[83139]: 2023-10-07T23:08:21.223+0000 7f85551261c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:08:21.630 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:21 smithi067 conmon[83139]: 2023-10-07T23:08:21.351+0000 7f85551261c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:08:21.891 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:21 smithi067 conmon[83139]: 2023-10-07T23:08:21.630+0000 7f85551261c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:08:22.196 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:21 smithi067 conmon[83139]: 2023-10-07T23:08:21.891+0000 7f85551261c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:08:22.196 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:22 smithi067 conmon[83139]: 2023-10-07T23:08:22.088+0000 7f85551261c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:08:22.504 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:22 smithi067 conmon[83139]: 2023-10-07T23:08:22.195+0000 7f85551261c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:08:23.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:22 smithi067 conmon[83139]: 2023-10-07T23:08:22.978+0000 7f85551261c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:08:24.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:23 smithi067 conmon[83139]: 2023-10-07T23:08:23.598+0000 7f85551261c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:08:24.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:23 smithi067 conmon[83139]: 2023-10-07T23:08:23.721+0000 7f85551261c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:08:24.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:23 smithi067 conmon[83139]: 2023-10-07T23:08:23.844+0000 7f85551261c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:08:24.978 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:24 smithi067 conmon[83139]: 2023-10-07T23:08:24.632+0000 7f85551261c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T23:08:24.979 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:24 smithi067 conmon[83139]: 2023-10-07T23:08:24.856+0000 7f85551261c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:08:25.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:24 smithi067 conmon[83139]: 2023-10-07T23:08:24.978+0000 7f85551261c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:08:25.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:25 smithi067 conmon[83139]: 2023-10-07T23:08:25.153+0000 7f85551261c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T23:08:25.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:25 smithi067 conmon[83139]: 2023-10-07T23:08:25.273+0000 7f85551261c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:08:26.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:26 smithi067 conmon[83139]: 2023-10-07T23:08:26.407+0000 7f85551261c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T23:08:26.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:26 smithi067 conmon[83139]: 2023-10-07T23:08:26.525+0000 7f85551261c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:08:26.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:26 smithi067 conmon[83139]: 2023-10-07T23:08:26.644+0000 7f85551261c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:08:27.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:26 smithi067 conmon[83139]: 2023-10-07T23:08:26.841+0000 7f85551261c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:08:27.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:26 smithi067 conmon[83139]: 2023-10-07T23:08:26.971+0000 7f85551261c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:08:27.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:27 smithi067 conmon[83139]: 2023-10-07T23:08:27.472+0000 7f85551261c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:08:27.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:27 smithi067 conmon[83139]: 2023-10-07T23:08:27.593+0000 7f85551261c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:08:27.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:27 smithi067 conmon[83139]: 2023-10-07T23:08:27.713+0000 7f85551261c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:08:28.254 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:08:27 smithi067 conmon[83139]: 2023-10-07T23:08:27.832+0000 7f85551261c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: Active manager daemon x restarted 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: Activating manager daemon x 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: osdmap e3: 0 total, 0 up, 0 in 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: mgrmap e10: x(active, starting, since 0.00365103s) 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-07T23:08:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: Manager daemon x is now available 2023-10-07T23:08:28.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:28.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: [07/Oct/2023:23:08:28] ENGINE Bus STARTING 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: [07/Oct/2023:23:08:28] ENGINE Serving on https://172.21.15.67:7150 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: [07/Oct/2023:23:08:28] ENGINE Bus STARTED 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:29.218 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:28 smithi067 ceph-mon[82723]: mgrmap e11: x(active, since 1.00834s) 2023-10-07T23:08:29.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2023-10-07T23:08:29.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-07T23:08:29.270 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.457+0000 7f4855d0a700 1 Processor -- start 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.457+0000 7f4855d0a700 1 -- start start 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.458+0000 7f4855d0a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f48500771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.458+0000 7f4855d0a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500790d0 con 0x7f4850078cc0 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.458+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f48500771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.458+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f48500771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44330/0 (socket says 172.21.15.67:44330) 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.458+0000 7f484f7fe700 1 -- 172.21.15.67:0/2502378110 learned_addr learned my addr 172.21.15.67:0/2502378110 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.458+0000 7f484f7fe700 1 -- 172.21.15.67:0/2502378110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4850077710 con 0x7f4850078cc0 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.459+0000 7f484f7fe700 1 --2- 172.21.15.67:0/2502378110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f48500771d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48400098e0 tx=0x7f4840009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f7aab7f7f438c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:29.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.459+0000 7f484e7fc700 1 -- 172.21.15.67:0/2502378110 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4840009e80 con 0x7f4850078cc0 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.459+0000 7f484e7fc700 1 -- 172.21.15.67:0/2502378110 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484000bd30 con 0x7f4850078cc0 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.459+0000 7f484e7fc700 1 -- 172.21.15.67:0/2502378110 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4840011ca0 con 0x7f4850078cc0 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 -- 172.21.15.67:0/2502378110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 msgr2=0x7f48500771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 --2- 172.21.15.67:0/2502378110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f48500771d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48400098e0 tx=0x7f4840009250 comp rx=0 tx=0).stop 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 -- 172.21.15.67:0/2502378110 shutdown_connections 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 --2- 172.21.15.67:0/2502378110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f48500771d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 -- 172.21.15.67:0/2502378110 >> 172.21.15.67:0/2502378110 conn(0x7f48500fda80 msgr2=0x7f48500ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 -- 172.21.15.67:0/2502378110 shutdown_connections 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.460+0000 7f4855d0a700 1 -- 172.21.15.67:0/2502378110 wait complete. 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f4855d0a700 1 Processor -- start 2023-10-07T23:08:29.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f4855d0a700 1 -- start start 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f4855d0a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f4850113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f4855d0a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48501137b0 con 0x7f4850078cc0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f4850113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f4850113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44334/0 (socket says 172.21.15.67:44334) 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.461+0000 7f484f7fe700 1 -- 172.21.15.67:0/3851581368 learned_addr learned my addr 172.21.15.67:0/3851581368 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.462+0000 7f484f7fe700 1 -- 172.21.15.67:0/3851581368 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400094f0 con 0x7f4850078cc0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.462+0000 7f484f7fe700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f4850113270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4840006fd0 tx=0x7f4840009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.462+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f484001a710 con 0x7f4850078cc0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.462+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484001ad40 con 0x7f4850078cc0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.462+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4840022c90 con 0x7f4850078cc0 2023-10-07T23:08:29.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.462+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48501139b0 con 0x7f4850078cc0 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.463+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4850113dc0 con 0x7f4850078cc0 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.464+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) v1 ==== 43278+0+0 (secure 0 0 0) 0x7f484001a870 con 0x7f4850078cc0 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.464+0000 7f4837fff700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.464+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 --> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4838039220 con 0x7f4838036660 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.464+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f484004a6d0 con 0x7f4850078cc0 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.464+0000 7f484effd700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.464+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:08:29.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.665+0000 7f484effd700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:21.665+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:22.066+0000 7f484effd700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:22.066+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:22.867+0000 7f484effd700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:22.867+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:24.468+0000 7f484effd700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:24.468+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:27.669+0000 7f484effd700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2928618291 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:27.669+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:27.836+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 10) v1 ==== 43045+0+0 (secure 0 0 0) 0x7f484002c960 con 0x7f4850078cc0 2023-10-07T23:08:29.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:27.836+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 msgr2=0x7f4838038b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:27.836+0000 7f4837fff700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.839+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 11) v1 ==== 43172+0+0 (secure 0 0 0) 0x7f484002bc00 con 0x7f4850078cc0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.839+0000 7f4837fff700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4838039fd0 0x7f483803c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.840+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4838039220 con 0x7f4838039fd0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.840+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4838039fd0 0x7f483803c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.841+0000 7f484effd700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4838039fd0 0x7f483803c3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f48440039d0 tx=0x7f48440092f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.842+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f4838039220 con 0x7f4838039fd0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.849+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f485004bb80 con 0x7f4838039fd0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.849+0000 7f4837fff700 1 -- 172.21.15.67:0/3851581368 <== mgr.14152 v2:172.21.15.67:6800/2299203628 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f485004bb80 con 0x7f4838039fd0 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4838039fd0 msgr2=0x7f483803c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:29.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4838039fd0 0x7f483803c3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f48440039d0 tx=0x7f48440092f0 comp rx=0 tx=0).stop 2023-10-07T23:08:29.277 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 msgr2=0x7f4850113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:29.277 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f4850113270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4840006fd0 tx=0x7f4840009250 comp rx=0 tx=0).stop 2023-10-07T23:08:29.277 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 shutdown_connections 2023-10-07T23:08:29.277 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4838039fd0 0x7f483803c3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:29.277 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4850078cc0 0x7f4850113270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:29.277 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 --2- 172.21.15.67:0/3851581368 >> [v2:172.21.15.67:6800/2928618291,v1:172.21.15.67:6801/2928618291] conn(0x7f4838036660 0x7f4838038b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:29.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 >> 172.21.15.67:0/3851581368 conn(0x7f48500fda80 msgr2=0x7f48500fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:29.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 shutdown_connections 2023-10-07T23:08:29.278 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:28.850+0000 7f4855d0a700 1 -- 172.21.15.67:0/3851581368 wait complete. 2023-10-07T23:08:29.278 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2023-10-07T23:08:29.278 INFO:teuthology.orchestra.run.smithi067.stdout:Generating a dashboard self-signed certificate... 2023-10-07T23:08:30.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:29 smithi067 ceph-mon[82723]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-07T23:08:30.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:29 smithi067 ceph-mon[82723]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-07T23:08:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-07T23:08:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.840+0000 7f67ef49c700 1 Processor -- start 2023-10-07T23:08:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.840+0000 7f67ef49c700 1 -- start start 2023-10-07T23:08:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.840+0000 7f67ef49c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e80771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.840+0000 7f67ef49c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e80790d0 con 0x7f67e8078cc0 2023-10-07T23:08:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.841+0000 7f67ed238700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e80771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.841+0000 7f67ed238700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e80771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52454/0 (socket says 172.21.15.67:52454) 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.841+0000 7f67ed238700 1 -- 172.21.15.67:0/732225079 learned_addr learned my addr 172.21.15.67:0/732225079 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.841+0000 7f67ed238700 1 -- 172.21.15.67:0/732225079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8077710 con 0x7f67e8078cc0 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.841+0000 7f67ed238700 1 --2- 172.21.15.67:0/732225079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e80771d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67e40098e0 tx=0x7f67e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b100bde88b67a175 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67dffff700 1 -- 172.21.15.67:0/732225079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e4009e80 con 0x7f67e8078cc0 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67dffff700 1 -- 172.21.15.67:0/732225079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67e400bd30 con 0x7f67e8078cc0 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67dffff700 1 -- 172.21.15.67:0/732225079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e4011ca0 con 0x7f67e8078cc0 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67ef49c700 1 -- 172.21.15.67:0/732225079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 msgr2=0x7f67e80771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67ef49c700 1 --2- 172.21.15.67:0/732225079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e80771d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67e40098e0 tx=0x7f67e4009250 comp rx=0 tx=0).stop 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67ef49c700 1 -- 172.21.15.67:0/732225079 shutdown_connections 2023-10-07T23:08:30.544 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67ef49c700 1 --2- 172.21.15.67:0/732225079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e80771d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.842+0000 7f67ef49c700 1 -- 172.21.15.67:0/732225079 >> 172.21.15.67:0/732225079 conn(0x7f67e80fda80 msgr2=0x7f67e80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.843+0000 7f67ef49c700 1 -- 172.21.15.67:0/732225079 shutdown_connections 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.843+0000 7f67ef49c700 1 -- 172.21.15.67:0/732225079 wait complete. 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.843+0000 7f67ef49c700 1 Processor -- start 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.843+0000 7f67ef49c700 1 -- start start 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ef49c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e8113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ef49c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81137b0 con 0x7f67e8078cc0 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ed238700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e8113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ed238700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e8113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52464/0 (socket says 172.21.15.67:52464) 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ed238700 1 -- 172.21.15.67:0/2536232818 learned_addr learned my addr 172.21.15.67:0/2536232818 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ed238700 1 -- 172.21.15.67:0/2536232818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40094f0 con 0x7f67e8078cc0 2023-10-07T23:08:30.545 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.844+0000 7f67ed238700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e8113270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67e4011930 tx=0x7f67e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.845+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e401a790 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.845+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67e401adc0 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.845+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e4022c10 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.845+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e81139b0 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.845+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e8113dc0 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.846+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 11) v1 ==== 43172+0+0 (secure 0 0 0) 0x7f67e40224c0 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.846+0000 7f67ddffb700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f67d4036540 0x7f67d40389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.846+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e804bb80 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.847+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f67e4049be0 con 0x7f67e8078cc0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.847+0000 7f67eca37700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f67d4036540 0x7f67d40389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.847+0000 7f67eca37700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f67d4036540 0x7f67d40389f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009990 tx=0x7f67d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:30.546 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.856+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f67e401e020 con 0x7f67e8078cc0 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:29.893+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 12) v1 ==== 43278+0+0 (secure 0 0 0) 0x7f67e402b350 con 0x7f67e8078cc0 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.077+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f67e80fef10 con 0x7f67d4036540 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.147+0000 7f67ddffb700 1 -- 172.21.15.67:0/2536232818 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f67e80fef10 con 0x7f67d4036540 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.149+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f67d4036540 msgr2=0x7f67d40389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.149+0000 7f67ef49c700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f67d4036540 0x7f67d40389f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009990 tx=0x7f67d8006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.149+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 msgr2=0x7f67e8113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.149+0000 7f67ef49c700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e8113270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67e4011930 tx=0x7f67e4009250 comp rx=0 tx=0).stop 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.150+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 shutdown_connections 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.150+0000 7f67ef49c700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f67d4036540 0x7f67d40389f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.150+0000 7f67ef49c700 1 --2- 172.21.15.67:0/2536232818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67e8078cc0 0x7f67e8113270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:30.547 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.150+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 >> 172.21.15.67:0/2536232818 conn(0x7f67e80fda80 msgr2=0x7f67e80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:30.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.150+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 shutdown_connections 2023-10-07T23:08:30.548 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:30.150+0000 7f67ef49c700 1 -- 172.21.15.67:0/2536232818 wait complete. 2023-10-07T23:08:30.548 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2023-10-07T23:08:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:30 smithi067 ceph-mon[82723]: mgrmap e12: x(active, since 2s) 2023-10-07T23:08:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:30 smithi067 ceph-mon[82723]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:30 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:30 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:31.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:30 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$/KUgqa23gD49lOoIBlqgNOPzSSqA/iSp/h3fMf0SpxhaleAzBj2bW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696720111, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.118+0000 7f2dc8e35700 1 Processor -- start 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.118+0000 7f2dc8e35700 1 -- start start 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.119+0000 7f2dc8e35700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.119+0000 7f2dc8e35700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4102c70 con 0x7f2dc4102320 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.119+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:32.017 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.119+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52478/0 (socket says 172.21.15.67:52478) 2023-10-07T23:08:32.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.119+0000 7f2dc259c700 1 -- 172.21.15.67:0/1450492191 learned_addr learned my addr 172.21.15.67:0/1450492191 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:32.018 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.119+0000 7f2dc259c700 1 -- 172.21.15.67:0/1450492191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4102db0 con 0x7f2dc4102320 2023-10-07T23:08:32.022 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.120+0000 7f2dc259c700 1 --2- 172.21.15.67:0/1450492191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4102730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2db40098e0 tx=0x7f2db4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6adf39b31f3f75 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:32.022 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.120+0000 7f2dc159a700 1 -- 172.21.15.67:0/1450492191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db4009e80 con 0x7f2dc4102320 2023-10-07T23:08:32.022 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.120+0000 7f2dc159a700 1 -- 172.21.15.67:0/1450492191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db400bd30 con 0x7f2dc4102320 2023-10-07T23:08:32.022 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.120+0000 7f2dc159a700 1 -- 172.21.15.67:0/1450492191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db4011ca0 con 0x7f2dc4102320 2023-10-07T23:08:32.022 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.121+0000 7f2dc8e35700 1 -- 172.21.15.67:0/1450492191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 msgr2=0x7f2dc4102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:32.022 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.121+0000 7f2dc8e35700 1 --2- 172.21.15.67:0/1450492191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4102730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2db40098e0 tx=0x7f2db4009250 comp rx=0 tx=0).stop 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.121+0000 7f2dc8e35700 1 -- 172.21.15.67:0/1450492191 shutdown_connections 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.121+0000 7f2dc8e35700 1 --2- 172.21.15.67:0/1450492191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4102730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.121+0000 7f2dc8e35700 1 -- 172.21.15.67:0/1450492191 >> 172.21.15.67:0/1450492191 conn(0x7f2dc40fda80 msgr2=0x7f2dc40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.121+0000 7f2dc8e35700 1 -- 172.21.15.67:0/1450492191 shutdown_connections 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.122+0000 7f2dc8e35700 1 -- 172.21.15.67:0/1450492191 wait complete. 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.122+0000 7f2dc8e35700 1 Processor -- start 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.122+0000 7f2dc8e35700 1 -- start start 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.123+0000 7f2dc8e35700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4111030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.123+0000 7f2dc8e35700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4111570 con 0x7f2dc4102320 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.123+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4111030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:32.023 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.123+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4111030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52486/0 (socket says 172.21.15.67:52486) 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.123+0000 7f2dc259c700 1 -- 172.21.15.67:0/3888500261 learned_addr learned my addr 172.21.15.67:0/3888500261 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.123+0000 7f2dc259c700 1 -- 172.21.15.67:0/3888500261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db40094f0 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.124+0000 7f2dc259c700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4111030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2db4000c00 tx=0x7f2db4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.124+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db401a760 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.124+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc4111770 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.124+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db401ad90 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.124+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db4022c60 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.124+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc4111b80 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.125+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc404bb80 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.126+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 43278+0+0 (secure 0 0 0) 0x7f2db402c440 con 0x7f2dc4102320 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.126+0000 7f2db2ffd700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2dac036660 0x7f2dac038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:32.024 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.126+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2db404a8f0 con 0x7f2dc4102320 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.126+0000 7f2dc1d9b700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2dac036660 0x7f2dac038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.127+0000 7f2dc1d9b700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2dac036660 0x7f2dac038b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009990 tx=0x7f2db8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.131+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2db4022570 con 0x7f2dc4102320 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.356+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2dc405f090 con 0x7f2dac036660 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.603+0000 7f2db2ffd700 1 -- 172.21.15.67:0/3888500261 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f2dc405f090 con 0x7f2dac036660 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2dac036660 msgr2=0x7f2dac038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2dac036660 0x7f2dac038b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009990 tx=0x7f2db8006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:32.025 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 msgr2=0x7f2dc4111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4111030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2db4000c00 tx=0x7f2db4009250 comp rx=0 tx=0).stop 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 shutdown_connections 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2dac036660 0x7f2dac038b10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 --2- 172.21.15.67:0/3888500261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dc4102320 0x7f2dc4111030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.605+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 >> 172.21.15.67:0/3888500261 conn(0x7f2dc40fda80 msgr2=0x7f2dc40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.606+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 shutdown_connections 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:31.606+0000 7f2dc8e35700 1 -- 172.21.15.67:0/3888500261 wait complete. 2023-10-07T23:08:32.026 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2023-10-07T23:08:32.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:31 smithi067 ceph-mon[82723]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:32.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:32.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:33.072 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1802452827' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-07T23:08:33.401 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2023-10-07T23:08:33.401 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.566+0000 7fcf3985c700 1 Processor -- start 2023-10-07T23:08:33.401 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.566+0000 7fcf3985c700 1 -- start start 2023-10-07T23:08:33.401 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.567+0000 7fcf3985c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf340771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.567+0000 7fcf3985c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340790d0 con 0x7fcf34078cc0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.567+0000 7fcf32ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf340771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.567+0000 7fcf32ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf340771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42428/0 (socket says 172.21.15.67:42428) 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.567+0000 7fcf32ffd700 1 -- 172.21.15.67:0/826038756 learned_addr learned my addr 172.21.15.67:0/826038756 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.567+0000 7fcf32ffd700 1 -- 172.21.15.67:0/826038756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34077710 con 0x7fcf34078cc0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.568+0000 7fcf32ffd700 1 --2- 172.21.15.67:0/826038756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf340771d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf240098e0 tx=0x7fcf24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d63c61ced3a77f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.568+0000 7fcf327fc700 1 -- 172.21.15.67:0/826038756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf24009e80 con 0x7fcf34078cc0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.568+0000 7fcf327fc700 1 -- 172.21.15.67:0/826038756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf2400bd30 con 0x7fcf34078cc0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.568+0000 7fcf327fc700 1 -- 172.21.15.67:0/826038756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf24011ca0 con 0x7fcf34078cc0 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.568+0000 7fcf3985c700 1 -- 172.21.15.67:0/826038756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 msgr2=0x7fcf340771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.568+0000 7fcf3985c700 1 --2- 172.21.15.67:0/826038756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf340771d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf240098e0 tx=0x7fcf24009250 comp rx=0 tx=0).stop 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.569+0000 7fcf3985c700 1 -- 172.21.15.67:0/826038756 shutdown_connections 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.569+0000 7fcf3985c700 1 --2- 172.21.15.67:0/826038756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf340771d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.569+0000 7fcf3985c700 1 -- 172.21.15.67:0/826038756 >> 172.21.15.67:0/826038756 conn(0x7fcf340fda80 msgr2=0x7fcf340ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.569+0000 7fcf3985c700 1 -- 172.21.15.67:0/826038756 shutdown_connections 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.569+0000 7fcf3985c700 1 -- 172.21.15.67:0/826038756 wait complete. 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.570+0000 7fcf3985c700 1 Processor -- start 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.570+0000 7fcf3985c700 1 -- start start 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf3985c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf34113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf3985c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34113770 con 0x7fcf34078cc0 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf32ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf34113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf32ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf34113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42432/0 (socket says 172.21.15.67:42432) 2023-10-07T23:08:33.403 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf32ffd700 1 -- 172.21.15.67:0/1802452827 learned_addr learned my addr 172.21.15.67:0/1802452827 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf32ffd700 1 -- 172.21.15.67:0/1802452827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240094f0 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.571+0000 7fcf32ffd700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf34113230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf24011930 tx=0x7fcf24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.572+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf2401a790 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.572+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf2401adc0 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.572+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf34113970 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.572+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf24022c10 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.572+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf34113d80 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.573+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3404bb80 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.573+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 43278+0+0 (secure 0 0 0) 0x7fcf240193c0 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.574+0000 7fcf2b7fe700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf20036660 0x7fcf20038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.574+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcf24049e40 con 0x7fcf34078cc0 2023-10-07T23:08:33.404 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.574+0000 7fcf2bfff700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf20036660 0x7fcf20038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.575+0000 7fcf2bfff700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf20036660 0x7fcf20038b10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009990 tx=0x7fcf1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.579+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fcf240108a0 con 0x7fcf34078cc0 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.795+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fcf341148b0 con 0x7fcf34078cc0 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.796+0000 7fcf2b7fe700 1 -- 172.21.15.67:0/1802452827 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fcf2401e030 con 0x7fcf34078cc0 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf20036660 msgr2=0x7fcf20038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf20036660 0x7fcf20038b10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009990 tx=0x7fcf1c006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 msgr2=0x7fcf34113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf34113230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf24011930 tx=0x7fcf24009250 comp rx=0 tx=0).stop 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 shutdown_connections 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf20036660 0x7fcf20038b10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 --2- 172.21.15.67:0/1802452827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf34078cc0 0x7fcf34113230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:33.405 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.798+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 >> 172.21.15.67:0/1802452827 conn(0x7fcf340fda80 msgr2=0x7fcf340fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:33.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.799+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 shutdown_connections 2023-10-07T23:08:33.406 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:32.799+0000 7fcf3985c700 1 -- 172.21.15.67:0/1802452827 wait complete. 2023-10-07T23:08:33.424 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-07T23:08:33.424 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2023-10-07T23:08:33.443 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-07T23:08:33.443 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2023-10-07T23:08:33.443 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2023-10-07T23:08:33.443 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-07T23:08:33.447 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2023-10-07T23:08:33.447 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:33.447 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2023-10-07T23:08:33.447 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2023-10-07T23:08:33.448 INFO:teuthology.orchestra.run.smithi067.stdout: Password: g042sqrfqf 2023-10-07T23:08:33.448 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:33.448 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config directory 2023-10-07T23:08:33.449 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling autotune for osd_memory_target 2023-10-07T23:08:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.305+0000 7f2b23678700 1 Processor -- start 2023-10-07T23:08:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.305+0000 7f2b23678700 1 -- start start 2023-10-07T23:08:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.305+0000 7f2b23678700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:35.083 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.305+0000 7f2b23678700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c105180 con 0x7f2b1c1047a0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.306+0000 7f2b21414700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.306+0000 7f2b21414700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42438/0 (socket says 172.21.15.67:42438) 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.306+0000 7f2b21414700 1 -- 172.21.15.67:0/3276384067 learned_addr learned my addr 172.21.15.67:0/3276384067 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.306+0000 7f2b21414700 1 -- 172.21.15.67:0/3276384067 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c1052c0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.306+0000 7f2b21414700 1 --2- 172.21.15.67:0/3276384067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c104bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2b180098e0 tx=0x7f2b18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a5f6bbc73d432a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b13fff700 1 -- 172.21.15.67:0/3276384067 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b18009e80 con 0x7f2b1c1047a0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b13fff700 1 -- 172.21.15.67:0/3276384067 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1800bd30 con 0x7f2b1c1047a0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b13fff700 1 -- 172.21.15.67:0/3276384067 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b18011ca0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 -- 172.21.15.67:0/3276384067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 msgr2=0x7f2b1c104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 --2- 172.21.15.67:0/3276384067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c104bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2b180098e0 tx=0x7f2b18009250 comp rx=0 tx=0).stop 2023-10-07T23:08:35.084 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 -- 172.21.15.67:0/3276384067 shutdown_connections 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 --2- 172.21.15.67:0/3276384067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c104bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 -- 172.21.15.67:0/3276384067 >> 172.21.15.67:0/3276384067 conn(0x7f2b1c077ab0 msgr2=0x7f2b1c077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 -- 172.21.15.67:0/3276384067 shutdown_connections 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.307+0000 7f2b23678700 1 -- 172.21.15.67:0/3276384067 wait complete. 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.308+0000 7f2b23678700 1 Processor -- start 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.308+0000 7f2b23678700 1 -- start start 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b23678700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b23678700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1136f0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b21414700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b21414700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c1131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42454/0 (socket says 172.21.15.67:42454) 2023-10-07T23:08:35.085 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b21414700 1 -- 172.21.15.67:0/2724189928 learned_addr learned my addr 172.21.15.67:0/2724189928 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b21414700 1 -- 172.21.15.67:0/2724189928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b180094f0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.309+0000 7f2b21414700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c1131b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b18000c00 tx=0x7f2b18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.310+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b1801a760 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.310+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c113950 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.310+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1801ad90 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.310+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b18022b80 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.310+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c114590 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.311+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 43278+0+0 (secure 0 0 0) 0x7f2b18022490 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.311+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c05f270 con 0x7f2b1c1047a0 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.311+0000 7f2b11ffb700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b08036660 0x7f2b08038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:35.086 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.311+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2b1804a5b0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.312+0000 7f2b20c13700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b08036660 0x7f2b08038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.312+0000 7f2b20c13700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b08036660 0x7f2b08038b10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009990 tx=0x7f2b0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.316+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2b180219f0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.518+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2b1c04bb80 con 0x7f2b1c1047a0 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.518+0000 7f2b11ffb700 1 -- 172.21.15.67:0/2724189928 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2b180213a0 con 0x7f2b1c1047a0 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b08036660 msgr2=0x7f2b08038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b08036660 0x7f2b08038b10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009990 tx=0x7f2b0c006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 msgr2=0x7f2b1c1131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c1131b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b18000c00 tx=0x7f2b18009250 comp rx=0 tx=0).stop 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 shutdown_connections 2023-10-07T23:08:35.087 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b08036660 0x7f2b08038b10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 --2- 172.21.15.67:0/2724189928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b1c1047a0 0x7f2b1c1131b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.520+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 >> 172.21.15.67:0/2724189928 conn(0x7f2b1c077ab0 msgr2=0x7f2b1c102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.521+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 shutdown_connections 2023-10-07T23:08:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:34.521+0000 7f2b23678700 1 -- 172.21.15.67:0/2724189928 wait complete. 2023-10-07T23:08:36.000 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:35 smithi067 ceph-mon[82723]: mgrmap e13: x(active, since 6s) 2023-10-07T23:08:36.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.646+0000 7f51275a4700 1 Processor -- start 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.646+0000 7f51275a4700 1 -- start start 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.647+0000 7f51275a4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f5120102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.647+0000 7f51275a4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120102c70 con 0x7f5120102320 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.647+0000 7f5125340700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f5120102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.647+0000 7f5125340700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f5120102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42462/0 (socket says 172.21.15.67:42462) 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.647+0000 7f5125340700 1 -- 172.21.15.67:0/1884208816 learned_addr learned my addr 172.21.15.67:0/1884208816 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.647+0000 7f5125340700 1 -- 172.21.15.67:0/1884208816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5120102db0 con 0x7f5120102320 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.648+0000 7f5125340700 1 --2- 172.21.15.67:0/1884208816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f5120102730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f511c0098e0 tx=0x7f511c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb952b0d0e29ecf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.648+0000 7f5117fff700 1 -- 172.21.15.67:0/1884208816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f511c009e80 con 0x7f5120102320 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.648+0000 7f5117fff700 1 -- 172.21.15.67:0/1884208816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f511c00bd30 con 0x7f5120102320 2023-10-07T23:08:36.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.648+0000 7f5117fff700 1 -- 172.21.15.67:0/1884208816 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f511c011ca0 con 0x7f5120102320 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 -- 172.21.15.67:0/1884208816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 msgr2=0x7f5120102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 --2- 172.21.15.67:0/1884208816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f5120102730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f511c0098e0 tx=0x7f511c009250 comp rx=0 tx=0).stop 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 -- 172.21.15.67:0/1884208816 shutdown_connections 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 --2- 172.21.15.67:0/1884208816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f5120102730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 -- 172.21.15.67:0/1884208816 >> 172.21.15.67:0/1884208816 conn(0x7f51200fda80 msgr2=0x7f51200ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 -- 172.21.15.67:0/1884208816 shutdown_connections 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.649+0000 7f51275a4700 1 -- 172.21.15.67:0/1884208816 wait complete. 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f51275a4700 1 Processor -- start 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f51275a4700 1 -- start start 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f51275a4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f51201131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:36.360 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f51275a4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120113730 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f5125340700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f51201131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f5125340700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f51201131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42472/0 (socket says 172.21.15.67:42472) 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.650+0000 7f5125340700 1 -- 172.21.15.67:0/2467958964 learned_addr learned my addr 172.21.15.67:0/2467958964 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f5125340700 1 -- 172.21.15.67:0/2467958964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0094f0 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f5125340700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f51201131f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f511c000c00 tx=0x7f511c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f511c01a760 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f511c01ad90 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f511c022c60 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120113930 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.651+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120113d40 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.652+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f511c022570 con 0x7f5120102320 2023-10-07T23:08:36.361 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.652+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512004bb80 con 0x7f5120102320 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.653+0000 7f5115ffb700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f510c036700 0x7f510c038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.653+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f511c04a920 con 0x7f5120102320 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.653+0000 7f5124b3f700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f510c036700 0x7f510c038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.653+0000 7f5124b3f700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f510c036700 0x7f510c038bb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5110009990 tx=0x7f5110006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.658+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f511c02ab80 con 0x7f5120102320 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.973+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f5120104220 con 0x7f5120102320 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.976+0000 7f5115ffb700 1 -- 172.21.15.67:0/2467958964 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f511c02ab80 con 0x7f5120102320 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f510c036700 msgr2=0x7f510c038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f510c036700 0x7f510c038bb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5110009990 tx=0x7f5110006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 msgr2=0x7f51201131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:36.362 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f51201131f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f511c000c00 tx=0x7f511c009250 comp rx=0 tx=0).stop 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 shutdown_connections 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f510c036700 0x7f510c038bb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 --2- 172.21.15.67:0/2467958964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5120102320 0x7f51201131f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 >> 172.21.15.67:0/2467958964 conn(0x7f51200fda80 msgr2=0x7f51200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 shutdown_connections 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2023-10-07T23:08:35.979+0000 7f51275a4700 1 -- 172.21.15.67:0/2467958964 wait complete. 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:08:36.363 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2023-10-07T23:08:36.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:36.365 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2023-10-07T23:08:36.415 INFO:tasks.cephadm:Fetching config... 2023-10-07T23:08:36.416 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:08:36.416 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-07T23:08:36.453 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-07T23:08:36.453 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:08:36.453 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-07T23:08:36.532 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-07T23:08:36.532 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:08:36.532 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-07T23:08:36.637 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-07T23:08:36.638 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:08:36.638 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-07T23:08:36.715 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-07T23:08:36.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-07T23:08:36.864 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:08:36.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-07T23:08:36.936 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:08:36.948 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-07T23:08:36.990 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/xDThbfGjkPhUiA9HAky4CoaVSWH4OByT6BrGSTBVcfLf9fNODSz1nLAZGKgRkMwn5kDraoOsgNipQCyWNzv8Fm2zYUbvWYd0OCkuLBYh2GwLoKUWbw4LlaUBI+h3U+DDWnx9rXYIh8HXIDbO8W4Nm8AIduxXCStyEg5IAE35l5OhzEf79lYTWULh40JBjZqoJqX6auZexLUrxBXgZiTsFNX2FKLdrt6uz/86/ufLJMjIuAD/j99aYGUzRwwph/2fFT95+pViuoOMhpcwIDWoDG8JMl9x/GWsWzBYJjcw8NW3lpx3NJva6TdKcednjv29P3RT5aNg4+7ILU5QzIN/Y4abvFprC9Oy72llVnRgbvMPYanXLUL2SUofNLQnAwiqCIhsenYABIKWgfAyXGTZW6/eaLwLXK8mk7wfxnPi5v8HtQHcOEDDeMCEy+mZnJMkMThPk3vDnKnuXEbluySJX6jv4zfumTzLwaEooOJ1b/MBnZ6ewu0OkegqC0sB4g8= ceph-227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:08:37.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-07T23:08:37.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:36 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2467958964' entity='client.admin' 2023-10-07T23:08:37.770 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.407+0000 7f000010e700 1 -- 172.21.15.67:0/1425931653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 msgr2=0x7efff806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.407+0000 7f000010e700 1 --2- 172.21.15.67:0/1425931653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff806ffb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efff40048c0 tx=0x7efff4009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.408+0000 7f000010e700 1 -- 172.21.15.67:0/1425931653 shutdown_connections 2023-10-07T23:08:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.408+0000 7f000010e700 1 --2- 172.21.15.67:0/1425931653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff806ffb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:39.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.408+0000 7f000010e700 1 -- 172.21.15.67:0/1425931653 >> 172.21.15.67:0/1425931653 conn(0x7efff80f7f40 msgr2=0x7efff80fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:39.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.408+0000 7f000010e700 1 -- 172.21.15.67:0/1425931653 shutdown_connections 2023-10-07T23:08:39.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.408+0000 7f000010e700 1 -- 172.21.15.67:0/1425931653 wait complete. 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.408+0000 7f000010e700 1 Processor -- start 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7f000010e700 1 -- start start 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7f000010e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff810ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7f000010e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff810f200 con 0x7efff8071b40 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7efffdeaa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff810ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7efffdeaa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff810ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42488/0 (socket says 172.21.15.67:42488) 2023-10-07T23:08:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7efffdeaa700 1 -- 172.21.15.67:0/3560106053 learned_addr learned my addr 172.21.15.67:0/3560106053 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.409+0000 7efffdeaa700 1 -- 172.21.15.67:0/3560106053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4005040 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.410+0000 7efffdeaa700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff810ecc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efff4008f00 tx=0x7efff4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.410+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff400e040 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.410+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff4013420 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.410+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff401c3f0 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.410+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff810f400 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.410+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff810ff60 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.411+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7efff4013590 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.411+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff804bb80 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.412+0000 7effee7fc700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7effe40366b0 0x7effe4038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.412+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7efff40495c0 con 0x7efff8071b40 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.412+0000 7efffd6a9700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7effe40366b0 0x7effe4038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.413+0000 7efffd6a9700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7effe40366b0 0x7effe4038b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7effe8009990 tx=0x7effe8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:39.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.417+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7efff4011960 con 0x7efff8071b40 2023-10-07T23:08:39.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.620+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7efff8110b50 con 0x7efff8071b40 2023-10-07T23:08:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.623+0000 7effee7fc700 1 -- 172.21.15.67:0/3560106053 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7efff4011370 con 0x7efff8071b40 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7effe40366b0 msgr2=0x7effe4038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7effe40366b0 0x7effe4038b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7effe8009990 tx=0x7effe8006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 msgr2=0x7efff810ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff810ecc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efff4008f00 tx=0x7efff4009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 shutdown_connections 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7effe40366b0 0x7effe4038b60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 --2- 172.21.15.67:0/3560106053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efff8071b40 0x7efff810ecc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 >> 172.21.15.67:0/3560106053 conn(0x7efff80f7f40 msgr2=0x7efff80f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 shutdown_connections 2023-10-07T23:08:39.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:39.626+0000 7f000010e700 1 -- 172.21.15.67:0/3560106053 wait complete. 2023-10-07T23:08:39.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3560106053' entity='client.admin' 2023-10-07T23:08:39.994 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:39.995 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:40.050 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-07T23:08:40.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-07T23:08:40.816 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:42.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 -- 172.21.15.67:0/2162077621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 msgr2=0x7f57200fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:42.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 --2- 172.21.15.67:0/2162077621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f57200fcc80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57100048c0 tx=0x7f5710009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:42.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 -- 172.21.15.67:0/2162077621 shutdown_connections 2023-10-07T23:08:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 --2- 172.21.15.67:0/2162077621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f57200fcc80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 -- 172.21.15.67:0/2162077621 >> 172.21.15.67:0/2162077621 conn(0x7f57200f7f50 msgr2=0x7f57200fa360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 -- 172.21.15.67:0/2162077621 shutdown_connections 2023-10-07T23:08:42.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.187+0000 7f57262d9700 1 -- 172.21.15.67:0/2162077621 wait complete. 2023-10-07T23:08:42.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.188+0000 7f57262d9700 1 Processor -- start 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.188+0000 7f57262d9700 1 -- start start 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f57262d9700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f5720113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f57262d9700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5720113570 con 0x7f57200fc870 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f571ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f5720113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f571ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f5720113030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53264/0 (socket says 172.21.15.67:53264) 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f571ffff700 1 -- 172.21.15.67:0/2393837389 learned_addr learned my addr 172.21.15.67:0/2393837389 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f571ffff700 1 -- 172.21.15.67:0/2393837389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5710005040 con 0x7f57200fc870 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.189+0000 7f571ffff700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f5720113030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5710008f30 tx=0x7f5710009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:42.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.190+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f571000e040 con 0x7f57200fc870 2023-10-07T23:08:42.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.190+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5710013420 con 0x7f57200fc870 2023-10-07T23:08:42.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.190+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5720113770 con 0x7f57200fc870 2023-10-07T23:08:42.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.190+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f571001c3f0 con 0x7f57200fc870 2023-10-07T23:08:42.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.190+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5720114410 con 0x7f57200fc870 2023-10-07T23:08:42.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.191+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f5710013590 con 0x7f57200fc870 2023-10-07T23:08:42.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.191+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572005f270 con 0x7f57200fc870 2023-10-07T23:08:42.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.191+0000 7f571cff9700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5708036700 0x7f5708038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:42.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.191+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5710049540 con 0x7f57200fc870 2023-10-07T23:08:42.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.192+0000 7f571f7fe700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5708036700 0x7f5708038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:42.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.192+0000 7f571f7fe700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5708036700 0x7f5708038bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f5714006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:42.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.196+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f5710011550 con 0x7f57200fc870 2023-10-07T23:08:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.404+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f5720114f50 con 0x7f5708036700 2023-10-07T23:08:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.408+0000 7f571cff9700 1 -- 172.21.15.67:0/2393837389 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5720114f50 con 0x7f5708036700 2023-10-07T23:08:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5708036700 msgr2=0x7f5708038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5708036700 0x7f5708038bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f5714006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 msgr2=0x7f5720113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f5720113030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5710008f30 tx=0x7f5710009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 shutdown_connections 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5708036700 0x7f5708038bb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 --2- 172.21.15.67:0/2393837389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57200fc870 0x7f5720113030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 >> 172.21.15.67:0/2393837389 conn(0x7f57200f7f50 msgr2=0x7f57200f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 shutdown_connections 2023-10-07T23:08:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:42.410+0000 7f57262d9700 1 -- 172.21.15.67:0/2393837389 wait complete. 2023-10-07T23:08:42.883 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi086 2023-10-07T23:08:42.884 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:08:42.884 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.conf 2023-10-07T23:08:42.901 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:08:42.901 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:08:42.957 INFO:tasks.cephadm:Adding host smithi086 to orchestrator... 2023-10-07T23:08:42.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi086 2023-10-07T23:08:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:08:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:08:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: Updating smithi067:/etc/ceph/ceph.conf 2023-10-07T23:08:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: Updating smithi067:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:08:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:43 smithi067 ceph-mon[82723]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 -- 172.21.15.67:0/1643765055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 msgr2=0x7fe9c4104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 --2- 172.21.15.67:0/1643765055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4104e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40048c0 tx=0x7fe9b4009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 -- 172.21.15.67:0/1643765055 shutdown_connections 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 --2- 172.21.15.67:0/1643765055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4104e90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 -- 172.21.15.67:0/1643765055 >> 172.21.15.67:0/1643765055 conn(0x7fe9c40fa600 msgr2=0x7fe9c40fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 -- 172.21.15.67:0/1643765055 shutdown_connections 2023-10-07T23:08:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.225+0000 7fe9c89c4700 1 -- 172.21.15.67:0/1643765055 wait complete. 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.226+0000 7fe9c89c4700 1 Processor -- start 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.226+0000 7fe9c89c4700 1 -- start start 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.226+0000 7fe9c89c4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.226+0000 7fe9c89c4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41151a0 con 0x7fe9c4104ac0 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9c259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9c259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4114c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53278/0 (socket says 172.21.15.67:53278) 2023-10-07T23:08:45.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9c259c700 1 -- 172.21.15.67:0/3569400606 learned_addr learned my addr 172.21.15.67:0/3569400606 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9c259c700 1 -- 172.21.15.67:0/3569400606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4005040 con 0x7fe9c4104ac0 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9c259c700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4114c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400b600 tx=0x7fe9b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9b400e040 con 0x7fe9c4104ac0 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9b40098c0 con 0x7fe9c4104ac0 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c41153a0 con 0x7fe9c4104ac0 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.227+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9b4013de0 con 0x7fe9c4104ac0 2023-10-07T23:08:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.228+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c4115890 con 0x7fe9c4104ac0 2023-10-07T23:08:45.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.229+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c405f270 con 0x7fe9c4104ac0 2023-10-07T23:08:45.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.229+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7fe9b40135b0 con 0x7fe9c4104ac0 2023-10-07T23:08:45.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.229+0000 7fe9b2ffd700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe9ac0366b0 0x7fe9ac038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:45.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.229+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe9b4049660 con 0x7fe9c4104ac0 2023-10-07T23:08:45.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.229+0000 7fe9c1d9b700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe9ac0366b0 0x7fe9ac038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:45.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.230+0000 7fe9c1d9b700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe9ac0366b0 0x7fe9ac038b60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:45.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.234+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fe9b4023300 con 0x7fe9c4104ac0 2023-10-07T23:08:45.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:45.442+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9c41102f0 con 0x7fe9ac0366b0 2023-10-07T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:45 smithi067 ceph-mon[82723]: Updating smithi067:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:45 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:45 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:45 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:46.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:46 smithi067 ceph-mon[82723]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:46.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:46 smithi067 ceph-mon[82723]: Deploying cephadm binary to smithi086 2023-10-07T23:08:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.336+0000 7fe9b2ffd700 1 -- 172.21.15.67:0/3569400606 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe9c41102f0 con 0x7fe9ac0366b0 2023-10-07T23:08:47.337 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi086' with addr '172.21.15.86' 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe9ac0366b0 msgr2=0x7fe9ac038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe9ac0366b0 0x7fe9ac038b60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b8006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 msgr2=0x7fe9c4114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4114c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400b600 tx=0x7fe9b4009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 shutdown_connections 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe9ac0366b0 0x7fe9ac038b60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 --2- 172.21.15.67:0/3569400606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9c4104ac0 0x7fe9c4114c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 >> 172.21.15.67:0/3569400606 conn(0x7fe9c40fa600 msgr2=0x7fe9c40fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 shutdown_connections 2023-10-07T23:08:47.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:47.339+0000 7fe9c89c4700 1 -- 172.21.15.67:0/3569400606 wait complete. 2023-10-07T23:08:47.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-07T23:08:48.579 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:48.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:48.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:48 smithi067 ceph-mon[82723]: Added host smithi086 2023-10-07T23:08:48.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:48.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:49 smithi067 ceph-mon[82723]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:08:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1151769131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 msgr2=0x7f3f580703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 --2- 172.21.15.67:0/1151769131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f580703b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0048c0 tx=0x7f3f4c009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1151769131 shutdown_connections 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 --2- 172.21.15.67:0/1151769131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f580703b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1151769131 >> 172.21.15.67:0/1151769131 conn(0x7f3f580f83f0 msgr2=0x7f3f580fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1151769131 shutdown_connections 2023-10-07T23:08:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.982+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1151769131 wait complete. 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5f04c700 1 Processor -- start 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5f04c700 1 -- start start 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5f04c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f58114d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5f04c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58115290 con 0x7f3f5806ffe0 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5cde8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f58114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5cde8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f58114d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53292/0 (socket says 172.21.15.67:53292) 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.983+0000 7f3f5cde8700 1 -- 172.21.15.67:0/1744482127 learned_addr learned my addr 172.21.15.67:0/1744482127 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f5cde8700 1 -- 172.21.15.67:0/1744482127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c005040 con 0x7f3f5806ffe0 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f5cde8700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f58114d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c005010 tx=0x7f3f4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f4c00e040 con 0x7f3f5806ffe0 2023-10-07T23:08:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f4c009890 con 0x7f3f5806ffe0 2023-10-07T23:08:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f4c013e20 con 0x7f3f5806ffe0 2023-10-07T23:08:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58115490 con 0x7f3f5806ffe0 2023-10-07T23:08:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.984+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58115900 con 0x7f3f5806ffe0 2023-10-07T23:08:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.985+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5805f270 con 0x7f3f5806ffe0 2023-10-07T23:08:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.985+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f3f4c0093c0 con 0x7f3f5806ffe0 2023-10-07T23:08:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.986+0000 7f3f557fa700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3f48036700 0x7f3f48038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.986+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3f4c049690 con 0x7f3f5806ffe0 2023-10-07T23:08:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.986+0000 7f3f57fff700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3f48036700 0x7f3f48038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.986+0000 7f3f57fff700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3f48036700 0x7f3f48038bb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009990 tx=0x7f3f44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:49.991+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f3f4c011960 con 0x7f3f5806ffe0 2023-10-07T23:08:50.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.199+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f581103a0 con 0x7f3f48036700 2023-10-07T23:08:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.200+0000 7f3f557fa700 1 -- 172.21.15.67:0/1744482127 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f3f581103a0 con 0x7f3f48036700 2023-10-07T23:08:50.200 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:50.201 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}] 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3f48036700 msgr2=0x7f3f48038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3f48036700 0x7f3f48038bb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009990 tx=0x7f3f44006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 msgr2=0x7f3f58114d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f58114d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c005010 tx=0x7f3f4c009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 shutdown_connections 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3f48036700 0x7f3f48038bb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 --2- 172.21.15.67:0/1744482127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f5806ffe0 0x7f3f58114d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 >> 172.21.15.67:0/1744482127 conn(0x7f3f580f83f0 msgr2=0x7f3f581033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 shutdown_connections 2023-10-07T23:08:50.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:50.202+0000 7f3f5f04c700 1 -- 172.21.15.67:0/1744482127 wait complete. 2023-10-07T23:08:50.644 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2023-10-07T23:08:50.644 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:08:50.644 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2023-10-07T23:08:50.661 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:08:50.661 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:08:50.717 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2023-10-07T23:08:50.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi123 2023-10-07T23:08:51.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:51.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:50 smithi067 ceph-mon[82723]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:08:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:50 smithi067 ceph-mon[82723]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:08:51.483 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 -- 172.21.15.67:0/3930047011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 msgr2=0x7fbf24104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:52.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 --2- 172.21.15.67:0/3930047011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24104e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbf140048c0 tx=0x7fbf14009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:52.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 -- 172.21.15.67:0/3930047011 shutdown_connections 2023-10-07T23:08:52.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 --2- 172.21.15.67:0/3930047011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24104e90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:52.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 -- 172.21.15.67:0/3930047011 >> 172.21.15.67:0/3930047011 conn(0x7fbf240fa600 msgr2=0x7fbf240fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 -- 172.21.15.67:0/3930047011 shutdown_connections 2023-10-07T23:08:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.883+0000 7fbf288ce700 1 -- 172.21.15.67:0/3930047011 wait complete. 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf288ce700 1 Processor -- start 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf288ce700 1 -- start start 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf288ce700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf288ce700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241151a0 con 0x7fbf24104ac0 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24114c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43762/0 (socket says 172.21.15.67:43762) 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.884+0000 7fbf2259c700 1 -- 172.21.15.67:0/3696820790 learned_addr learned my addr 172.21.15.67:0/3696820790 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf2259c700 1 -- 172.21.15.67:0/3696820790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14005040 con 0x7fbf24104ac0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf2259c700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24114c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b600 tx=0x7fbf14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf1400e040 con 0x7fbf24104ac0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf140098c0 con 0x7fbf24104ac0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf241153a0 con 0x7fbf24104ac0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf14013de0 con 0x7fbf24104ac0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.885+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf24115890 con 0x7fbf24104ac0 2023-10-07T23:08:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.886+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7fbf140135b0 con 0x7fbf24104ac0 2023-10-07T23:08:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.886+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2405f270 con 0x7fbf24104ac0 2023-10-07T23:08:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.887+0000 7fbf12ffd700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fbf0c036700 0x7fbf0c038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.887+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbf140495e0 con 0x7fbf24104ac0 2023-10-07T23:08:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.887+0000 7fbf21d9b700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fbf0c036700 0x7fbf0c038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.888+0000 7fbf21d9b700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fbf0c036700 0x7fbf0c038bb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009990 tx=0x7fbf18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:52.892+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fbf14023300 con 0x7fbf24104ac0 2023-10-07T23:08:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:53.104+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf2410eb40 con 0x7fbf0c036700 2023-10-07T23:08:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:52 smithi067 ceph-mon[82723]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:08:54.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: Updating smithi086:/etc/ceph/ceph.conf 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:54.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.888+0000 7fbf12ffd700 1 -- 172.21.15.67:0/3696820790 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbf2410eb40 con 0x7fbf0c036700 2023-10-07T23:08:54.889 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi123' with addr '172.21.15.123' 2023-10-07T23:08:54.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fbf0c036700 msgr2=0x7fbf0c038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:54.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fbf0c036700 0x7fbf0c038bb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009990 tx=0x7fbf18006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 msgr2=0x7fbf24114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24114c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b600 tx=0x7fbf14009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 shutdown_connections 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fbf0c036700 0x7fbf0c038bb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 --2- 172.21.15.67:0/3696820790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf24104ac0 0x7fbf24114c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 >> 172.21.15.67:0/3696820790 conn(0x7fbf240fa600 msgr2=0x7fbf240fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 shutdown_connections 2023-10-07T23:08:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:54.891+0000 7fbf288ce700 1 -- 172.21.15.67:0/3696820790 wait complete. 2023-10-07T23:08:55.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:55 smithi067 ceph-mon[82723]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:08:55.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:55 smithi067 ceph-mon[82723]: Deploying cephadm binary to smithi123 2023-10-07T23:08:55.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:55 smithi067 ceph-mon[82723]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:08:55.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:55 smithi067 ceph-mon[82723]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:08:55.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:55 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:55.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:55 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:08:55.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-07T23:08:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:56.686 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:56 smithi067 ceph-mon[82723]: Added host smithi123 2023-10-07T23:08:56.686 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:56.686 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.482+0000 7ffba1b30700 1 -- 172.21.15.67:0/3355228845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 msgr2=0x7ffb9c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.482+0000 7ffba1b30700 1 --2- 172.21.15.67:0/3355228845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c102ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0048c0 tx=0x7ffb8c009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.482+0000 7ffba1b30700 1 -- 172.21.15.67:0/3355228845 shutdown_connections 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.482+0000 7ffba1b30700 1 --2- 172.21.15.67:0/3355228845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c102ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.482+0000 7ffba1b30700 1 -- 172.21.15.67:0/3355228845 >> 172.21.15.67:0/3355228845 conn(0x7ffb9c0f8410 msgr2=0x7ffb9c0fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.482+0000 7ffba1b30700 1 -- 172.21.15.67:0/3355228845 shutdown_connections 2023-10-07T23:08:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.483+0000 7ffba1b30700 1 -- 172.21.15.67:0/3355228845 wait complete. 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.483+0000 7ffba1b30700 1 Processor -- start 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.484+0000 7ffba1b30700 1 -- start start 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.484+0000 7ffba1b30700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.484+0000 7ffba1b30700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c112fc0 con 0x7ffb9c1028d0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.484+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.484+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43780/0 (socket says 172.21.15.67:43780) 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.484+0000 7ffb9b7fe700 1 -- 172.21.15.67:0/4097916301 learned_addr learned my addr 172.21.15.67:0/4097916301 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffb9b7fe700 1 -- 172.21.15.67:0/4097916301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c005040 con 0x7ffb9c1028d0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffb9b7fe700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c112a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b600 tx=0x7ffb8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb8c00e040 con 0x7ffb9c1028d0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb8c0098c0 con 0x7ffb9c1028d0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c1131c0 con 0x7ffb9c1028d0 2023-10-07T23:08:57.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb8c013de0 con 0x7ffb9c1028d0 2023-10-07T23:08:57.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.485+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c10c940 con 0x7ffb9c1028d0 2023-10-07T23:08:57.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.486+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7ffb8c0093f0 con 0x7ffb9c1028d0 2023-10-07T23:08:57.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.486+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c05f270 con 0x7ffb9c1028d0 2023-10-07T23:08:57.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.487+0000 7ffb83fff700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffb84036700 0x7ffb84038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:08:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.487+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffb8c049600 con 0x7ffb9c1028d0 2023-10-07T23:08:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.487+0000 7ffb9affd700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffb84036700 0x7ffb84038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:08:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.488+0000 7ffb9affd700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffb84036700 0x7ffb84038bb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009990 tx=0x7ffb90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:08:57.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.492+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7ffb8c0119c0 con 0x7ffb9c1028d0 2023-10-07T23:08:57.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:57 smithi067 ceph-mon[82723]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:08:57.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:57 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:08:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.698+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb9c112750 con 0x7ffb84036700 2023-10-07T23:08:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.699+0000 7ffb83fff700 1 -- 172.21.15.67:0/4097916301 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7ffb9c112750 con 0x7ffb84036700 2023-10-07T23:08:57.700 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:08:57.700 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2023-10-07T23:08:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.701+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffb84036700 msgr2=0x7ffb84038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.701+0000 7ffba1b30700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffb84036700 0x7ffb84038bb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009990 tx=0x7ffb90006e40 comp rx=0 tx=0).stop 2023-10-07T23:08:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.701+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 msgr2=0x7ffb9c112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:08:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.701+0000 7ffba1b30700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c112a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b600 tx=0x7ffb8c009e30 comp rx=0 tx=0).stop 2023-10-07T23:08:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.702+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 shutdown_connections 2023-10-07T23:08:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.702+0000 7ffba1b30700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffb84036700 0x7ffb84038bb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.702+0000 7ffba1b30700 1 --2- 172.21.15.67:0/4097916301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb9c1028d0 0x7ffb9c112a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:08:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.702+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 >> 172.21.15.67:0/4097916301 conn(0x7ffb9c0f8410 msgr2=0x7ffb9c0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:08:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.702+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 shutdown_connections 2023-10-07T23:08:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:08:57.702+0000 7ffba1b30700 1 -- 172.21.15.67:0/4097916301 wait complete. 2023-10-07T23:08:58.086 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-07T23:08:58.087 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-07T23:08:58.855 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:08:59.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:59 smithi067 ceph-mon[82723]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:08:59.444 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:08:59 smithi067 ceph-mon[82723]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:00.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 -- 172.21.15.67:0/3803263403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 msgr2=0x7fab2c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:00.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 --2- 172.21.15.67:0/3803263403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c102ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0048c0 tx=0x7fab1c009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:00.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 -- 172.21.15.67:0/3803263403 shutdown_connections 2023-10-07T23:09:00.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 --2- 172.21.15.67:0/3803263403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c102ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:00.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 -- 172.21.15.67:0/3803263403 >> 172.21.15.67:0/3803263403 conn(0x7fab2c0f8410 msgr2=0x7fab2c0fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:00.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 -- 172.21.15.67:0/3803263403 shutdown_connections 2023-10-07T23:09:00.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.263+0000 7fab31106700 1 -- 172.21.15.67:0/3803263403 wait complete. 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.264+0000 7fab31106700 1 Processor -- start 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.264+0000 7fab31106700 1 -- start start 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab31106700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c114cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab31106700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c115210 con 0x7fab2c1028d0 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c114cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c114cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43806/0 (socket says 172.21.15.67:43806) 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab2ad9d700 1 -- 172.21.15.67:0/2905003881 learned_addr learned my addr 172.21.15.67:0/2905003881 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:09:00.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab2ad9d700 1 -- 172.21.15.67:0/2905003881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c005040 con 0x7fab2c1028d0 2023-10-07T23:09:00.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.265+0000 7fab2ad9d700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c114cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b600 tx=0x7fab1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:00.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.266+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1c00e040 con 0x7fab2c1028d0 2023-10-07T23:09:00.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.266+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c0098c0 con 0x7fab2c1028d0 2023-10-07T23:09:00.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.266+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1c013de0 con 0x7fab2c1028d0 2023-10-07T23:09:00.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.266+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c115410 con 0x7fab2c1028d0 2023-10-07T23:09:00.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.266+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c115900 con 0x7fab2c1028d0 2023-10-07T23:09:00.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.267+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c05f270 con 0x7fab2c1028d0 2023-10-07T23:09:00.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.267+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7fab1c0135b0 con 0x7fab2c1028d0 2023-10-07T23:09:00.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.267+0000 7fab137fe700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fab14036700 0x7fab14038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:00.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.268+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fab1c049520 con 0x7fab2c1028d0 2023-10-07T23:09:00.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.268+0000 7fab2a59c700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fab14036700 0x7fab14038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:00.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.268+0000 7fab2a59c700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fab14036700 0x7fab14038bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab20009990 tx=0x7fab20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:00.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.272+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fab1c023300 con 0x7fab2c1028d0 2023-10-07T23:09:00.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:00.482+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fab2c106ac0 con 0x7fab2c1028d0 2023-10-07T23:09:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.360+0000 7fab137fe700 1 -- 172.21.15.67:0/2905003881 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fab1c011b60 con 0x7fab2c1028d0 2023-10-07T23:09:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fab14036700 msgr2=0x7fab14038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fab14036700 0x7fab14038bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab20009990 tx=0x7fab20006e40 comp rx=0 tx=0).stop 2023-10-07T23:09:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 msgr2=0x7fab2c114cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c114cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b600 tx=0x7fab1c009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 shutdown_connections 2023-10-07T23:09:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fab14036700 0x7fab14038bb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 --2- 172.21.15.67:0/2905003881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c1028d0 0x7fab2c114cd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:01.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.364+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 >> 172.21.15.67:0/2905003881 conn(0x7fab2c0f8410 msgr2=0x7fab2c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:01.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.365+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 shutdown_connections 2023-10-07T23:09:01.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:01.365+0000 7fab31106700 1 -- 172.21.15.67:0/2905003881 wait complete. 2023-10-07T23:09:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2023-10-07T23:09:01.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:01.689 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2905003881' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:01.690 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:01.718 INFO:tasks.cephadm:Adding mon.a on smithi067 2023-10-07T23:09:01.718 INFO:tasks.cephadm:Adding mon.b on smithi086 2023-10-07T23:09:01.718 INFO:tasks.cephadm:Adding mon.c on smithi123 2023-10-07T23:09:01.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c' 2023-10-07T23:09:02.318 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:02.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: Updating smithi123:/etc/ceph/ceph.conf 2023-10-07T23:09:02.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2905003881' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-07T23:09:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:03 smithi067 ceph-mon[82723]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:09:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:03 smithi067 ceph-mon[82723]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:03.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 -- 172.21.15.123:0/2015789840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 msgr2=0x7f9ce8102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:03.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 --2- 172.21.15.123:0/2015789840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8102c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40048c0 tx=0x7f9ce4009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:03.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 -- 172.21.15.123:0/2015789840 shutdown_connections 2023-10-07T23:09:03.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 --2- 172.21.15.123:0/2015789840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8102c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 -- 172.21.15.123:0/2015789840 >> 172.21.15.123:0/2015789840 conn(0x7f9ce80f8410 msgr2=0x7f9ce80fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 -- 172.21.15.123:0/2015789840 shutdown_connections 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.795+0000 7f9cf0334700 1 -- 172.21.15.123:0/2015789840 wait complete. 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cf0334700 1 Processor -- start 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cf0334700 1 -- start start 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cf0334700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:03.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cf0334700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8113000 con 0x7f9ce81028b0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cee0d0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cee0d0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8112ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.123:58624/0 (socket says 172.21.15.123:58624) 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.796+0000 7f9cee0d0700 1 -- 172.21.15.123:0/2714115488 learned_addr learned my addr 172.21.15.123:0/2714115488 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cee0d0700 1 -- 172.21.15.123:0/2714115488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4005040 con 0x7f9ce81028b0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cee0d0700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8112ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4005010 tx=0x7f9ce4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce400e040 con 0x7f9ce81028b0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce8113200 con 0x7f9ce81028b0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce4009890 con 0x7f9ce81028b0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce4013e20 con 0x7f9ce81028b0 2023-10-07T23:09:03.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.797+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce810c980 con 0x7f9ce81028b0 2023-10-07T23:09:03.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.798+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce805f270 con 0x7f9ce81028b0 2023-10-07T23:09:03.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.798+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f9ce40093c0 con 0x7f9ce81028b0 2023-10-07T23:09:03.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.798+0000 7f9cdeffd700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9cd40366b0 0x7f9cd4038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:03.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.798+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9ce40495f0 con 0x7f9ce81028b0 2023-10-07T23:09:03.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.799+0000 7f9ced8cf700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9cd40366b0 0x7f9cd4038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:03.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.799+0000 7f9ced8cf700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9cd40366b0 0x7f9cd4038b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009990 tx=0x7f9cd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:03.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.804+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9ce4011bd0 con 0x7f9ce81028b0 2023-10-07T23:09:03.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:03.996+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ce810e0a0 con 0x7f9cd40366b0 2023-10-07T23:09:04.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.004+0000 7f9cdeffd700 1 -- 172.21.15.123:0/2714115488 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9ce810e0a0 con 0x7f9cd40366b0 2023-10-07T23:09:04.004 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mon update... 2023-10-07T23:09:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.005+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9cd40366b0 msgr2=0x7f9cd4038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.005+0000 7f9cf0334700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9cd40366b0 0x7f9cd4038b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009990 tx=0x7f9cd8006e40 comp rx=0 tx=0).stop 2023-10-07T23:09:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 msgr2=0x7f9ce8112ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8112ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4005010 tx=0x7f9ce4009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 shutdown_connections 2023-10-07T23:09:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9cd40366b0 0x7f9cd4038b60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:04.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 --2- 172.21.15.123:0/2714115488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce81028b0 0x7f9ce8112ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:04.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 >> 172.21.15.123:0/2714115488 conn(0x7f9ce80f8410 msgr2=0x7f9ce80f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:04.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 shutdown_connections 2023-10-07T23:09:04.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:04.006+0000 7f9cf0334700 1 -- 172.21.15.123:0/2714115488 wait complete. 2023-10-07T23:09:04.665 DEBUG:teuthology.orchestra.run.smithi086:mon.b> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-07T23:09:04.667 DEBUG:teuthology.orchestra.run.smithi123:mon.c> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-07T23:09:04.671 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-07T23:09:04.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-07T23:09:04.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:09:04.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:09:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:05 smithi067 ceph-mon[82723]: Deploying daemon mon.c on smithi123 2023-10-07T23:09:05.656 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:07.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:07 smithi067 ceph-mon[82723]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 -- 172.21.15.123:0/2170066669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 msgr2=0x7fec64101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 --2- 172.21.15.123:0/2170066669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec64101ba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fec540048c0 tx=0x7fec54009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 -- 172.21.15.123:0/2170066669 shutdown_connections 2023-10-07T23:09:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 --2- 172.21.15.123:0/2170066669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec64101ba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 -- 172.21.15.123:0/2170066669 >> 172.21.15.123:0/2170066669 conn(0x7fec640f72f0 msgr2=0x7fec640f9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:08.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 -- 172.21.15.123:0/2170066669 shutdown_connections 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.025+0000 7fec69718700 1 -- 172.21.15.123:0/2170066669 wait complete. 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.026+0000 7fec69718700 1 Processor -- start 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.026+0000 7fec69718700 1 -- start start 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.026+0000 7fec69718700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec641119d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.026+0000 7fec69718700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64111f10 con 0x7fec641017d0 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec62ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec641119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec62ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec641119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.123:58630/0 (socket says 172.21.15.123:58630) 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec62ffd700 1 -- 172.21.15.123:0/2224735118 learned_addr learned my addr 172.21.15.123:0/2224735118 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec62ffd700 1 -- 172.21.15.123:0/2224735118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54005040 con 0x7fec641017d0 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec62ffd700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec641119d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec54005010 tx=0x7fec54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:08.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec5400e040 con 0x7fec641017d0 2023-10-07T23:09:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec64112110 con 0x7fec641017d0 2023-10-07T23:09:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec54009650 con 0x7fec641017d0 2023-10-07T23:09:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec54013420 con 0x7fec641017d0 2023-10-07T23:09:08.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.027+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641125a0 con 0x7fec641017d0 2023-10-07T23:09:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.028+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7fec540136e0 con 0x7fec641017d0 2023-10-07T23:09:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.028+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6405f270 con 0x7fec641017d0 2023-10-07T23:09:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.028+0000 7fec4b7fe700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fec4c0366b0 0x7fec4c038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.028+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fec5404c130 con 0x7fec641017d0 2023-10-07T23:09:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.029+0000 7fec627fc700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fec4c0366b0 0x7fec4c038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:08.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.029+0000 7fec627fc700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fec4c0366b0 0x7fec4c038b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fec58009990 tx=0x7fec58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:08.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.034+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fec54026d30 con 0x7fec641017d0 2023-10-07T23:09:08.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:07 smithi123 systemd[1]: Starting Ceph mon.c for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:09:08.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.314+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fec6404bb80 con 0x7fec641017d0 2023-10-07T23:09:08.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.315+0000 7fec4b7fe700 1 -- 172.21.15.123:0/2224735118 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+731 (secure 0 0 0) 0x7fec54026b50 con 0x7fec641017d0 2023-10-07T23:09:08.315 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:09:08.315 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","modified":"2023-10-07T23:07:30.168632Z","created":"2023-10-07T23:07:30.168632Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-07T23:09:08.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fec4c0366b0 msgr2=0x7fec4c038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:08.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fec4c0366b0 0x7fec4c038b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fec58009990 tx=0x7fec58006e40 comp rx=0 tx=0).stop 2023-10-07T23:09:08.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 msgr2=0x7fec641119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:08.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec641119d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec54005010 tx=0x7fec54009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 shutdown_connections 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fec4c0366b0 0x7fec4c038b60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 --2- 172.21.15.123:0/2224735118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fec641017d0 0x7fec641119d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 >> 172.21.15.123:0/2224735118 conn(0x7fec640f72f0 msgr2=0x7fec640f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 shutdown_connections 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:08.317+0000 7fec69718700 1 -- 172.21.15.123:0/2224735118 wait complete. 2023-10-07T23:09:08.318 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: set uid:gid to 167:167 (ceph:ceph) 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable), process ceph-mon, pid 7 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pidfile_write: ignore empty --pid-file 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: load: jerasure load: lrc 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: RocksDB version: 6.15.5 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Compile date Oct 6 2023 2023-10-07T23:09:08.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: DB SUMMARY 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: DB Session ID: 6QU0OKLEB86DOSRCYJPD 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: CURRENT file: CURRENT 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: IDENTITY file: IDENTITY 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.error_if_exists: 0 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.create_if_missing: 0 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.paranoid_checks: 1 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-07T23:09:08.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.env: 0x55db07023a80 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.fs: Posix File System 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.info_log: 0x55db07fb7d80 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_file_opening_threads: 16 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.statistics: (nil) 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.use_fsync: 0 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_log_file_size: 0 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-07T23:09:08.538 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.keep_log_file_num: 1000 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.recycle_log_file_num: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.allow_fallocate: 1 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.allow_mmap_reads: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.allow_mmap_writes: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.use_direct_reads: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.create_missing_column_families: 0 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.db_log_dir: 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2023-10-07T23:09:08.539 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.advise_random_on_open: 1 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.db_write_buffer_size: 0 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.write_buffer_manager: 0x55db08caa960 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-07T23:09:08.540 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.rate_limiter: (nil) 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.wal_recovery_mode: 2 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.enable_thread_tracking: 0 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.enable_pipelined_write: 0 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.unordered_write: 0 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-07T23:09:08.541 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.row_cache: None 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.wal_filter: None 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.allow_ingest_behind: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.preserve_deletes: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.two_write_queues: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.manual_wal_flush: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.atomic_flush: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.log_readahead_size: 0 2023-10-07T23:09:08.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-07T23:09:08.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.best_efforts_recovery: 0 2023-10-07T23:09:08.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-07T23:09:08.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-07T23:09:08.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.allow_data_in_errors: 0 2023-10-07T23:09:08.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.db_host_id: __hostname__ 2023-10-07T23:09:08.544 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_background_jobs: 2 2023-10-07T23:09:08.544 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_background_compactions: -1 2023-10-07T23:09:08.544 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_subcompactions: 1 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_total_wal_size: 0 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_open_files: -1 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bytes_per_sync: 0 2023-10-07T23:09:08.545 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_readahead_size: 0 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_background_flushes: -1 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Compression algorithms supported: 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kZSTD supported: 0 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kXpressCompression supported: 0 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kLZ4HCCompression supported: 1 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kZlibCompression supported: 1 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kSnappyCompression supported: 1 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kLZ4Compression supported: 1 2023-10-07T23:09:08.546 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: kBZip2Compression supported: 0 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.merge_operator: 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_filter: None 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_filter_factory: None 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.sst_partitioner_factory: None 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55db07f8afa8) 2023-10-07T23:09:08.547 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks: 1 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_top_level_index_and_filter: 1 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_type: 0 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_index_type: 0 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_shortening: 1 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: hash_index_allow_collision: 1 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: checksum: 1 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: no_block_cache: 0 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache: 0x55db07fbd090 2023-10-07T23:09:08.548 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_name: BinnedLRUCache 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_options: 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: capacity : 536870912 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_shard_bits : 4 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: strict_capacity_limit : 0 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: high_pri_pool_ratio: 0.000 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_compressed: (nil) 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: persistent_cache: (nil) 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size: 4096 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size_deviation: 10 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_restart_interval: 16 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_block_restart_interval: 1 2023-10-07T23:09:08.549 INFO:journalctl@ceph.mon.c.smithi123.stdout: metadata_block_size: 4096 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: partition_filters: 0 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: use_delta_encoding: 1 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: whole_key_filtering: 1 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: verify_compression: 0 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: read_amp_bytes_per_bit: 0 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: format_version: 4 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_index_compression: 1 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_align: 0 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.write_buffer_size: 33554432 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_write_buffer_number: 2 2023-10-07T23:09:08.550 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression: NoCompression 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression: Disabled 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.prefix_extractor: nullptr 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.num_levels: 7 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-07T23:09:08.551 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.level: 32767 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.strategy: 0 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compression_opts.enabled: false 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-07T23:09:08.552 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.target_file_size_base: 67108864 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-07T23:09:08.553 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.arena_block_size: 4194304 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.disable_auto_compactions: 0 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-07T23:09:08.554 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.table_properties_collectors: 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.inplace_update_support: 0 2023-10-07T23:09:08.555 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.bloom_locality: 0 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.max_successive_merges: 0 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.paranoid_file_checks: 0 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.force_consistency_checks: 1 2023-10-07T23:09:08.557 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.report_bg_io_stats: 0 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.ttl: 2592000 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.enable_blob_files: false 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.min_blob_size: 0 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.blob_file_size: 268435456 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2023-10-07T23:09:08.558 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720148420568, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720148420947, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1696720148, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "cedd27b6-9f42-4b37-8b3e-93d372334420", "db_session_id": "6QU0OKLEB86DOSRCYJPD"}} 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720148421389, "job": 1, "event": "recovery_finished"} 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55db07f51180 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: DB pointer 0x55db08026000 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-10-07T23:09:08.559 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** DB Stats ** 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:08.560 INFO:journalctl@ceph.mon.c.smithi123.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:08.561 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:08.562 INFO:journalctl@ceph.mon.c.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:09:08.563 INFO:journalctl@ceph.mon.c.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: using public_addr v2:172.21.15.123:0/0 -> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2023-10-07T23:09:08.564 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: starting mon.c rank -1 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(???) e0 preinit fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).mds e1 new map 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout: e1 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout: legacy client fscid: -1 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout: No filesystems configured 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-07T23:09:08.565 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Deploying cephadm binary to smithi123 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Added host smithi123 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2905003881' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Updating smithi123:/etc/ceph/ceph.conf 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2905003881' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:08.569 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: Deploying daemon mon.c on smithi123 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 ceph-mon[122926]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 bash[122855]: 43a316b4d95240303b2503ebb5fc49d8a77eee9be2a4e852bbd4b1d93b2fc999 2023-10-07T23:09:08.570 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:08 smithi123 systemd[1]: Started Ceph mon.c for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:09:10.247 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-07T23:09:10.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-07T23:09:10.833 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:09:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 -- 172.21.15.123:0/3116327049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b8102970 msgr2=0x7f65b8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:13.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 --2- 172.21.15.123:0/3116327049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b8102970 0x7f65b8102d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009ce0 tx=0x7f65b400b040 comp rx=0 tx=0).stop 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 -- 172.21.15.123:0/3116327049 shutdown_connections 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 --2- 172.21.15.123:0/3116327049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b8102970 0x7f65b8102d40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 -- 172.21.15.123:0/3116327049 >> 172.21.15.123:0/3116327049 conn(0x7f65b80f84f0 msgr2=0x7f65b80fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 -- 172.21.15.123:0/3116327049 shutdown_connections 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.837+0000 7f65c0470700 1 -- 172.21.15.123:0/3116327049 wait complete. 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65c0470700 1 Processor -- start 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65c0470700 1 -- start start 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65c0470700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65c0470700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b81132e0 0x7f65b810cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65c0470700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8113860 con 0x7f65b81132e0 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65c0470700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b81139d0 con 0x7f65b8102970 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:13.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37574/0 (socket says 172.21.15.123:37574) 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 -- 172.21.15.123:0/1412885625 learned_addr learned my addr 172.21.15.123:0/1412885625 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 -- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 msgr2=0x7f65b8112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 -- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 msgr2=0x7f65b8112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.838+0000 7f65be20c700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.839+0000 7f65bda0b700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b81132e0 0x7f65b810cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.839+0000 7f65bda0b700 1 -- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 msgr2=0x7f65b8112da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.839+0000 7f65bda0b700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.839+0000 7f65bda0b700 1 -- 172.21.15.123:0/1412885625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4005040 con 0x7f65b81132e0 2023-10-07T23:09:13.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.839+0000 7f65bda0b700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b81132e0 0x7f65b810cc40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65a800b790 tx=0x7f65a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.839+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f65a80117e0 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.840+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65a8011e10 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.840+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b810d240 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.840+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f65a8010e80 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.840+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b810d800 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.841+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f65a801a400 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.841+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b810da80 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.841+0000 7f65aeffd700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f65a4036840 0x7f65a4038cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.841+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f65a8048840 con 0x7f65b81132e0 2023-10-07T23:09:13.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.841+0000 7f65be20c700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f65a4036840 0x7f65a4038cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:13.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.842+0000 7f65be20c700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f65a4036840 0x7f65a4038cf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65b4004730 tx=0x7f65b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:13.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:13.847+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f65a80174a0 con 0x7f65b81132e0 2023-10-07T23:09:13.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: mon.a calling monitor election 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: mon.c calling monitor election 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:13.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: fsmap 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: mgrmap e13: x(active, since 45s) 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: overall HEALTH_OK 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:13.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:14.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: mon.a calling monitor election 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: mon.c calling monitor election 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:14.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: fsmap 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: mgrmap e13: x(active, since 45s) 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: overall HEALTH_OK 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:14.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:14.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.110+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f65b805f270 con 0x7f65b81132e0 2023-10-07T23:09:14.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.110+0000 7f65aeffd700 1 -- 172.21.15.123:0/1412885625 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1001 (secure 0 0 0) 0x7f65a8015030 con 0x7f65b81132e0 2023-10-07T23:09:14.111 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:09:14.111 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","modified":"2023-10-07T23:09:08.446489Z","created":"2023-10-07T23:07:30.168632Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-07T23:09:14.112 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f65a4036840 msgr2=0x7f65a4038cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f65a4036840 0x7f65a4038cf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65b4004730 tx=0x7f65b400b040 comp rx=0 tx=0).stop 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b81132e0 msgr2=0x7f65b810cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b81132e0 0x7f65b810cc40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65a800b790 tx=0x7f65a80091a0 comp rx=0 tx=0).stop 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 shutdown_connections 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f65a4036840 0x7f65a4038cf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b8102970 0x7f65b8112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 --2- 172.21.15.123:0/1412885625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65b81132e0 0x7f65b810cc40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 >> 172.21.15.123:0/1412885625 conn(0x7f65b80f84f0 msgr2=0x7f65b80f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 shutdown_connections 2023-10-07T23:09:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:14.112+0000 7f65c0470700 1 -- 172.21.15.123:0/1412885625 wait complete. 2023-10-07T23:09:14.114 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2023-10-07T23:09:15.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:14 smithi067 ceph-mon[82723]: Deploying daemon mon.b on smithi086 2023-10-07T23:09:15.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:14 smithi067 ceph-mon[82723]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:15.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:14 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/1412885625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-07T23:09:15.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:14 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:15.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:14 smithi123 ceph-mon[122926]: Deploying daemon mon.b on smithi086 2023-10-07T23:09:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:14 smithi123 ceph-mon[122926]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:14 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/1412885625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-07T23:09:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:14 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:15.754 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:09:15 smithi067 conmon[83139]: 2023-10-07T23:09:15.446+0000 7f850e5c1700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-07T23:09:15.764 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-07T23:09:15.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-07T23:09:16.354 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:09:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 systemd[1]: Starting Ceph mon.b for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:09:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: set uid:gid to 167:167 (ceph:ceph) 2023-10-07T23:09:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable), process ceph-mon, pid 7 2023-10-07T23:09:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pidfile_write: ignore empty --pid-file 2023-10-07T23:09:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: load: jerasure load: lrc 2023-10-07T23:09:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: RocksDB version: 6.15.5 2023-10-07T23:09:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Compile date Oct 6 2023 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: DB SUMMARY 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 bash[122067]: 5f35a1f401f9075d10a541abcd6ef68792a2270eba56cc323103e0765bca37d2 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: DB Session ID: AITK8R5S9SYDKUME6V9V 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: CURRENT file: CURRENT 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: IDENTITY file: IDENTITY 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.error_if_exists: 0 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.create_if_missing: 0 2023-10-07T23:09:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.paranoid_checks: 1 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.env: 0x55c02d357a80 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.fs: Posix File System 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.info_log: 0x55c02e07dd80 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_file_opening_threads: 16 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.statistics: (nil) 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.use_fsync: 0 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_log_file_size: 0 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-07T23:09:17.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.keep_log_file_num: 1000 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.recycle_log_file_num: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.allow_fallocate: 1 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.allow_mmap_reads: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.allow_mmap_writes: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.use_direct_reads: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.create_missing_column_families: 0 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.db_log_dir: 2023-10-07T23:09:17.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.advise_random_on_open: 1 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.db_write_buffer_size: 0 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.write_buffer_manager: 0x55c02ed70900 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-07T23:09:17.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.rate_limiter: (nil) 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.wal_recovery_mode: 2 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.enable_thread_tracking: 0 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.enable_pipelined_write: 0 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.unordered_write: 0 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-07T23:09:17.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.row_cache: None 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.wal_filter: None 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.allow_ingest_behind: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.preserve_deletes: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.two_write_queues: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.manual_wal_flush: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.atomic_flush: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-07T23:09:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.log_readahead_size: 0 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.best_efforts_recovery: 0 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.allow_data_in_errors: 0 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.db_host_id: __hostname__ 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_background_jobs: 2 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_background_compactions: -1 2023-10-07T23:09:17.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_subcompactions: 1 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_total_wal_size: 0 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_open_files: -1 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bytes_per_sync: 0 2023-10-07T23:09:17.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_readahead_size: 0 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_background_flushes: -1 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Compression algorithms supported: 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kZSTD supported: 0 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kXpressCompression supported: 0 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kLZ4HCCompression supported: 1 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kZlibCompression supported: 1 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kSnappyCompression supported: 1 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kLZ4Compression supported: 1 2023-10-07T23:09:17.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: kBZip2Compression supported: 0 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.merge_operator: 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_filter: None 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_filter_factory: None 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.sst_partitioner_factory: None 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c02e050fa8) 2023-10-07T23:09:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks: 1 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_top_level_index_and_filter: 1 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_type: 0 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_index_type: 0 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_shortening: 1 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: hash_index_allow_collision: 1 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: checksum: 1 2023-10-07T23:09:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout: no_block_cache: 0 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache: 0x55c02e083090 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_name: BinnedLRUCache 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_options: 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: capacity : 536870912 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_shard_bits : 4 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: strict_capacity_limit : 0 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: high_pri_pool_ratio: 0.000 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_compressed: (nil) 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: persistent_cache: (nil) 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size: 4096 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size_deviation: 10 2023-10-07T23:09:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_restart_interval: 16 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_block_restart_interval: 1 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: metadata_block_size: 4096 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: partition_filters: 0 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: use_delta_encoding: 1 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: whole_key_filtering: 1 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: verify_compression: 0 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: read_amp_bytes_per_bit: 0 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: format_version: 4 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_index_compression: 1 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_align: 0 2023-10-07T23:09:17.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.write_buffer_size: 33554432 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_write_buffer_number: 2 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression: NoCompression 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression: Disabled 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.prefix_extractor: nullptr 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.num_levels: 7 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-07T23:09:17.213 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.level: 32767 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.strategy: 0 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-07T23:09:17.214 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compression_opts.enabled: false 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.target_file_size_base: 67108864 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-07T23:09:17.215 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.arena_block_size: 4194304 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-10-07T23:09:17.216 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.disable_auto_compactions: 0 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-07T23:09:17.217 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.table_properties_collectors: 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.inplace_update_support: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.bloom_locality: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.max_successive_merges: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.paranoid_file_checks: 0 2023-10-07T23:09:17.218 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.force_consistency_checks: 1 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.report_bg_io_stats: 0 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.ttl: 2592000 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.enable_blob_files: false 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.min_blob_size: 0 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.blob_file_size: 268435456 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2023-10-07T23:09:17.219 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720156875509, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720156875866, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1670, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1696720156, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "eb33119d-11d5-4f9c-8532-8991abfb65fe", "db_session_id": "AITK8R5S9SYDKUME6V9V"}} 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696720156876274, "job": 1, "event": "recovery_finished"} 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55c02e017180 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: DB pointer 0x55c02e0ec000 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-10-07T23:09:17.220 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** DB Stats ** 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:17.221 INFO:journalctl@ceph.mon.b.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:17.222 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:17.223 INFO:journalctl@ceph.mon.b.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:17.224 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: using public_addr v2:172.21.15.86:0/0 -> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] 2023-10-07T23:09:17.225 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: starting mon.b rank -1 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(???) e0 preinit fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).mds e1 new map 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout: e1 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout: legacy client fscid: -1 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout: No filesystems configured 2023-10-07T23:09:17.226 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:17.227 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Deploying cephadm binary to smithi123 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Added host smithi123 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.228 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.229 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2905003881' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-07T23:09:17.229 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.229 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.229 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.229 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Updating smithi123:/etc/ceph/ceph.conf 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2905003881' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.230 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:17.231 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Deploying daemon mon.c on smithi123 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.a calling monitor election 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.232 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.c calling monitor election 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: fsmap 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mgrmap e13: x(active, since 45s) 2023-10-07T23:09:17.233 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: overall HEALTH_OK 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: Deploying daemon mon.b on smithi086 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/1412885625' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 ceph-mon[122131]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-07T23:09:17.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:16 smithi086 systemd[1]: Started Ceph mon.b for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:09:22.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: mon.a calling monitor election 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: mon.c calling monitor election 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: fsmap 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: mgrmap e13: x(active, since 54s) 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: mon.b (rank 2) addr [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] is down (out of quorum) 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.257 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.257 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 -- 172.21.15.123:0/2300062890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884003580 msgr2=0x7f4884005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 --2- 172.21.15.123:0/2300062890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884003580 0x7f4884005a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f489c0fc8c0 tx=0x7f488c002040 comp rx=0 tx=0).stop 2023-10-07T23:09:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 -- 172.21.15.123:0/2300062890 shutdown_connections 2023-10-07T23:09:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 --2- 172.21.15.123:0/2300062890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884003580 0x7f4884005a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 -- 172.21.15.123:0/2300062890 >> 172.21.15.123:0/2300062890 conn(0x7f489c0f8550 msgr2=0x7f489c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:22.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 -- 172.21.15.123:0/2300062890 shutdown_connections 2023-10-07T23:09:22.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 -- 172.21.15.123:0/2300062890 wait complete. 2023-10-07T23:09:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 Processor -- start 2023-10-07T23:09:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.295+0000 7f48a2365700 1 -- start start 2023-10-07T23:09:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a2365700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a2365700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c070480 0x7f489c0708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a2365700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 0x7f489c11b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:22.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a2365700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c11ba00 con 0x7f489c070480 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a2365700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c11bb70 con 0x7f489c118e30 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a2365700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c11bd30 con 0x7f4884003580 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37602/0 (socket says 172.21.15.123:37602) 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 -- 172.21.15.123:0/3197205348 learned_addr learned my addr 172.21.15.123:0/3197205348 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 msgr2=0x7f489c06ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 msgr2=0x7f489c06ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489bfff700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a0902700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 0x7f489c11b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489b7fe700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c070480 0x7f489c0708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a0902700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 msgr2=0x7f489c11b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a0902700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 msgr2=0x7f489c11b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a0902700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 0x7f489c11b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f48a0902700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 0x7f489c11b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489b7fe700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 msgr2=0x7f489c06ff40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489b7fe700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489b7fe700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 msgr2=0x7f489c11b230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489b7fe700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 0x7f489c11b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.296+0000 7f489b7fe700 1 -- 172.21.15.123:0/3197205348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c005040 con 0x7f489c070480 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.297+0000 7f489b7fe700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c070480 0x7f489c0708f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f489000b810 tx=0x7f48900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.297+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890010670 con 0x7f489c070480 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.297+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4890010ca0 con 0x7f489c070480 2023-10-07T23:09:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.297+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c11c010 con 0x7f489c070480 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.297+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000fd40 con 0x7f489c070480 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.297+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c11c4d0 con 0x7f489c070480 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.298+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f489000d580 con 0x7f489c070480 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.298+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c05f270 con 0x7f489c070480 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.298+0000 7f4898ff9700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4888036b80 0x7f4888039030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.298+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4890048700 con 0x7f489c070480 2023-10-07T23:09:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.298+0000 7f489bfff700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4888036b80 0x7f4888039030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.299+0000 7f489bfff700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4888036b80 0x7f4888039030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f488c009040 tx=0x7f488c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.303+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4890017940 con 0x7f489c070480 2023-10-07T23:09:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: mon.a calling monitor election 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: mon.c calling monitor election 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: fsmap 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: mgrmap e13: x(active, since 54s) 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-07T23:09:22.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: mon.b (rank 2) addr [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] is down (out of quorum) 2023-10-07T23:09:22.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.579+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f489c1045d0 con 0x7f489c070480 2023-10-07T23:09:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.580+0000 7f4898ff9700 1 -- 172.21.15.123:0/3197205348 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1265 (secure 0 0 0) 0x7f489001c170 con 0x7f489c070480 2023-10-07T23:09:22.580 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-07T23:09:22.580 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","modified":"2023-10-07T23:09:16.898797Z","created":"2023-10-07T23:07:30.168632Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4888036b80 msgr2=0x7f4888039030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4888036b80 0x7f4888039030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f488c009040 tx=0x7f488c00b040 comp rx=0 tx=0).stop 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c070480 msgr2=0x7f489c0708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c070480 0x7f489c0708f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f489000b810 tx=0x7f48900091a0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 shutdown_connections 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f489c118e30 0x7f489c11b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4888036b80 0x7f4888039030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4884003580 0x7f489c06ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 --2- 172.21.15.123:0/3197205348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c070480 0x7f489c0708f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 >> 172.21.15.123:0/3197205348 conn(0x7f489c0f8550 msgr2=0x7f489c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 shutdown_connections 2023-10-07T23:09:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:22.582+0000 7f48a2365700 1 -- 172.21.15.123:0/3197205348 wait complete. 2023-10-07T23:09:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 3 2023-10-07T23:09:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:22 smithi067 ceph-mon[82723]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:22 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/3197205348' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-07T23:09:23.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:23.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:23.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:22 smithi123 ceph-mon[122926]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:23.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:23.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:22 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/3197205348' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-07T23:09:23.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:23.271 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:23.296 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-07T23:09:23.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-07T23:09:24.078 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:09:24.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: mon.a calling monitor election 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: mon.c calling monitor election 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: fsmap 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: mgrmap e13: x(active, since 54s) 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: mon.b (rank 2) addr [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] is down (out of quorum) 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/3197205348' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:24.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:25.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: mon.b calling monitor election 2023-10-07T23:09:25.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: mon.b calling monitor election 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: mon.c calling monitor election 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: mon.a calling monitor election 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: fsmap 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: mgrmap e13: x(active, since 56s) 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Cluster is now healthy 2023-10-07T23:09:25.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: overall HEALTH_OK 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Updating smithi067:/etc/ceph/ceph.conf 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Updating smithi086:/etc/ceph/ceph.conf 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Updating smithi123:/etc/ceph/ceph.conf 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.007 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: mon.b calling monitor election 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: mon.b calling monitor election 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: mon.c calling monitor election 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: mon.a calling monitor election 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: fsmap 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: mgrmap e13: x(active, since 56s) 2023-10-07T23:09:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Cluster is now healthy 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: overall HEALTH_OK 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Updating smithi067:/etc/ceph/ceph.conf 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Updating smithi086:/etc/ceph/ceph.conf 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Updating smithi123:/etc/ceph/ceph.conf 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:25.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:25.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: mon.b calling monitor election 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: mon.b calling monitor election 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: mon.c calling monitor election 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: mon.a calling monitor election 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: fsmap 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: osdmap e4: 0 total, 0 up, 0 in 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: mgrmap e13: x(active, since 56s) 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-07T23:09:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Cluster is now healthy 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: overall HEALTH_OK 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Updating smithi067:/etc/ceph/ceph.conf 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Updating smithi086:/etc/ceph/ceph.conf 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Updating smithi123:/etc/ceph/ceph.conf 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Updating smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: Updating smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:25.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 -- 172.21.15.67:0/2654502925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 msgr2=0x7f4664072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:25.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 --2- 172.21.15.67:0/2654502925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f4664072540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f46600048c0 tx=0x7f4660009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 -- 172.21.15.67:0/2654502925 shutdown_connections 2023-10-07T23:09:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 --2- 172.21.15.67:0/2654502925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f4664072540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 -- 172.21.15.67:0/2654502925 >> 172.21.15.67:0/2654502925 conn(0x7f46640fa630 msgr2=0x7f46640fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 -- 172.21.15.67:0/2654502925 shutdown_connections 2023-10-07T23:09:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.595+0000 7f466b26e700 1 -- 172.21.15.67:0/2654502925 wait complete. 2023-10-07T23:09:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.596+0000 7f466b26e700 1 Processor -- start 2023-10-07T23:09:25.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.596+0000 7f466b26e700 1 -- start start 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.596+0000 7f466b26e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f46641193f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466b26e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4664119930 0x7f46641132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466b26e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4664113820 0x7f4664113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466b26e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664119f90 con 0x7f4664072170 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466b26e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664120220 con 0x7f4664113820 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466b26e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664120360 con 0x7f4664119930 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f46641193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f46641193f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59008/0 (socket says 172.21.15.67:59008) 2023-10-07T23:09:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 -- 172.21.15.67:0/3962148687 learned_addr learned my addr 172.21.15.67:0/3962148687 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466980b700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4664113820 0x7f4664113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f4668809700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4664119930 0x7f46641132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 -- 172.21.15.67:0/3962148687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4664119930 msgr2=0x7f46641132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4664119930 0x7f46641132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 -- 172.21.15.67:0/3962148687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4664113820 msgr2=0x7f4664113cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4664113820 0x7f4664113cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 -- 172.21.15.67:0/3962148687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660005040 con 0x7f4664072170 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f4668809700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4664119930 0x7f46641132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.597+0000 7f466900a700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f46641193f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4660009000 tx=0x7f4660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.598+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000e040 con 0x7f4664072170 2023-10-07T23:09:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.598+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4660013420 con 0x7f4664072170 2023-10-07T23:09:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.598+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466001c660 con 0x7f4664072170 2023-10-07T23:09:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.598+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46641205b0 con 0x7f4664072170 2023-10-07T23:09:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.598+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4664120ad0 con 0x7f4664072170 2023-10-07T23:09:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.599+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466405f270 con 0x7f4664072170 2023-10-07T23:09:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.599+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f46600094f0 con 0x7f4664072170 2023-10-07T23:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.599+0000 7f4659ffb700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4650036b70 0x7f4650039020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.599+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f46600496f0 con 0x7f4664072170 2023-10-07T23:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.600+0000 7f4668809700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4650036b70 0x7f4650039020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.600+0000 7f4668809700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4650036b70 0x7f4650039020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4654007a80 tx=0x7f4654008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.605+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4660011550 con 0x7f4664072170 2023-10-07T23:09:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.816+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f466410d0b0 con 0x7f4664072170 2023-10-07T23:09:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.817+0000 7f4659ffb700 1 -- 172.21.15.67:0/3962148687 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f4660011370 con 0x7f4664072170 2023-10-07T23:09:25.818 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:09:25.818 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2023-10-07T23:09:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:09:25.818 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4650036b70 msgr2=0x7f4650039020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4650036b70 0x7f4650039020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4654007a80 tx=0x7f4654008040 comp rx=0 tx=0).stop 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 msgr2=0x7f46641193f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f46641193f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4660009000 tx=0x7f4660009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 shutdown_connections 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4664113820 0x7f4664113cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4650036b70 0x7f4650039020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4664072170 0x7f46641193f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 --2- 172.21.15.67:0/3962148687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4664119930 0x7f46641132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 >> 172.21.15.67:0/3962148687 conn(0x7f46640fa630 msgr2=0x7f46640faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 shutdown_connections 2023-10-07T23:09:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:25.819+0000 7f466b26e700 1 -- 172.21.15.67:0/3962148687 wait complete. 2023-10-07T23:09:26.004 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:09:25 smithi067 conmon[83139]: 2023-10-07T23:09:25.898+0000 7f850e5c1700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-07T23:09:26.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: Updating smithi067:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:26.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:26.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: Reconfiguring mon.a (unknown last config time)... 2023-10-07T23:09:26.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:26.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:26.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:26.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:25 smithi067 ceph-mon[82723]: Reconfiguring daemon mon.a on smithi067 2023-10-07T23:09:26.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: Updating smithi067:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: Reconfiguring mon.a (unknown last config time)... 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:25 smithi086 ceph-mon[122131]: Reconfiguring daemon mon.a on smithi067 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: Updating smithi067:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-07T23:09:26.200 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: Reconfiguring mon.a (unknown last config time)... 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:26.201 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:25 smithi123 ceph-mon[122926]: Reconfiguring daemon mon.a on smithi067 2023-10-07T23:09:26.259 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-07T23:09:26.383 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:09:26.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2023-10-07T23:09:26.455 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:09:26.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:26.562 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:09:26.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2023-10-07T23:09:26.598 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:09:26.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:26.666 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:09:26.666 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2023-10-07T23:09:26.698 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:09:26.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-07T23:09:26.764 INFO:tasks.cephadm:Adding mgr.x on smithi067 2023-10-07T23:09:26.765 INFO:tasks.cephadm:Adding mgr.y on smithi086 2023-10-07T23:09:26.765 INFO:tasks.cephadm:Adding mgr.z on smithi123 2023-10-07T23:09:26.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi067=x;smithi086=y;smithi123=z' 2023-10-07T23:09:26.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3962148687' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:26.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:26 smithi086 ceph-mon[122131]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:27.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3962148687' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:27.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:26 smithi067 ceph-mon[82723]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:27.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3962148687' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:27.173 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:26 smithi123 ceph-mon[122926]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:27.378 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:09:27.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:27.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:27.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: Reconfiguring mon.b (monmap changed)... 2023-10-07T23:09:27.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:27.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:27.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:27.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:27 smithi123 ceph-mon[122926]: Reconfiguring daemon mon.b on smithi086 2023-10-07T23:09:28.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:28.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:28.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: Reconfiguring mon.b (monmap changed)... 2023-10-07T23:09:28.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:28.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:28.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:28.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:27 smithi086 ceph-mon[122131]: Reconfiguring daemon mon.b on smithi086 2023-10-07T23:09:28.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:28.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:28.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: Reconfiguring mon.b (monmap changed)... 2023-10-07T23:09:28.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:28.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:28.270 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:28.271 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:27 smithi067 ceph-mon[82723]: Reconfiguring daemon mon.b on smithi086 2023-10-07T23:09:28.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 -- 172.21.15.123:0/112970872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 msgr2=0x7f56880fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:28.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 --2- 172.21.15.123:0/112970872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f56880fbcf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56840048c0 tx=0x7f5684009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:28.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 -- 172.21.15.123:0/112970872 shutdown_connections 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 --2- 172.21.15.123:0/112970872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f56880fbcf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 -- 172.21.15.123:0/112970872 >> 172.21.15.123:0/112970872 conn(0x7f56880f6fc0 msgr2=0x7f56880f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 -- 172.21.15.123:0/112970872 shutdown_connections 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.893+0000 7f568f934700 1 -- 172.21.15.123:0/112970872 wait complete. 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 Processor -- start 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 -- start start 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f568810ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 0x7f568810e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f568810e710 0x7f568810ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688115420 con 0x7f56880fb8e0 2023-10-07T23:09:28.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688115560 con 0x7f568810e710 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568f934700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881156a0 con 0x7f568810d410 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568d6d0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f568810ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 0x7f568810e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568d6d0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f568810ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.123:58696/0 (socket says 172.21.15.123:58696) 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 0x7f568810e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37614/0 (socket says 172.21.15.123:37614) 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568d6d0700 1 -- 172.21.15.123:0/2868535562 learned_addr learned my addr 172.21.15.123:0/2868535562 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568ded1700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f568810e710 0x7f568810ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 -- 172.21.15.123:0/2868535562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f568810e710 msgr2=0x7f568810ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f568810e710 0x7f568810ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 -- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 msgr2=0x7f568810ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:28.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f568810ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.894+0000 7f568cecf700 1 -- 172.21.15.123:0/2868535562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684005040 con 0x7f568810d410 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.895+0000 7f568cecf700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 0x7f568810e1d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f567800b810 tx=0x7f56780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.895+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678011630 con 0x7f568810d410 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.895+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5678011c60 con 0x7f568810d410 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.895+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56881158d0 con 0x7f568810d410 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.895+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678010d30 con 0x7f568810d410 2023-10-07T23:09:28.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.895+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5688115e40 con 0x7f568810d410 2023-10-07T23:09:28.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.896+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7f567800f4f0 con 0x7f568810d410 2023-10-07T23:09:28.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.896+0000 7f567dffb700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5674036b20 0x7f5674038fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:28.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.896+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5678048640 con 0x7f568810d410 2023-10-07T23:09:28.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.896+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568804bb80 con 0x7f568810d410 2023-10-07T23:09:28.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.896+0000 7f568d6d0700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5674036b20 0x7f5674038fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:28.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.897+0000 7f568d6d0700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5674036b20 0x7f5674038fd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f568400b3b0 tx=0x7f5684009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:28.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:28.901+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f567801a400 con 0x7f568810d410 2023-10-07T23:09:29.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.104+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi086=y;smithi123=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f5688065330 con 0x7f5674036b20 2023-10-07T23:09:29.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.111+0000 7f567dffb700 1 -- 172.21.15.123:0/2868535562 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5688065330 con 0x7f5674036b20 2023-10-07T23:09:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mgr update... 2023-10-07T23:09:29.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5674036b20 msgr2=0x7f5674038fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5674036b20 0x7f5674038fd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f568400b3b0 tx=0x7f5684009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 msgr2=0x7f568810e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 0x7f568810e1d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f567800b810 tx=0x7f56780091a0 comp rx=0 tx=0).stop 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 shutdown_connections 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f568810e710 0x7f568810ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5674036b20 0x7f5674038fd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56880fb8e0 0x7f568810ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 --2- 172.21.15.123:0/2868535562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568810d410 0x7f568810e1d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 >> 172.21.15.123:0/2868535562 conn(0x7f56880f6fc0 msgr2=0x7f56880f7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 shutdown_connections 2023-10-07T23:09:29.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:09:29.113+0000 7f568f934700 1 -- 172.21.15.123:0/2868535562 wait complete. 2023-10-07T23:09:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: Reconfiguring mon.c (monmap changed)... 2023-10-07T23:09:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: Reconfiguring daemon mon.c on smithi123 2023-10-07T23:09:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: Reconfiguring mon.c (monmap changed)... 2023-10-07T23:09:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: Reconfiguring daemon mon.c on smithi123 2023-10-07T23:09:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: Reconfiguring mon.c (monmap changed)... 2023-10-07T23:09:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-07T23:09:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-07T23:09:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: Reconfiguring daemon mon.c on smithi123 2023-10-07T23:09:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:30.311 DEBUG:teuthology.orchestra.run.smithi086:mgr.y> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-07T23:09:30.314 DEBUG:teuthology.orchestra.run.smithi123:mgr.z> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-07T23:09:30.316 INFO:tasks.cephadm:Deploying OSDs... 2023-10-07T23:09:30.317 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:09:30.317 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T23:09:30.342 INFO:journalctl@ceph.mgr.z.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:09:30.345 INFO:journalctl@ceph.mgr.y.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:09:30.358 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T23:09:30.358 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2023-10-07T23:09:30.438 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T23:09:30.438 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-07 23:08:37.637877626 +0000 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-07 23:02:57.971330963 +0000 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-07 23:02:57.971330963 +0000 2023-10-07T23:09:30.439 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-07T23:09:30.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T23:09:30.540 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-07T23:09:30.540 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-07T23:09:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000297461 s, 1.7 MB/s 2023-10-07T23:09:30.543 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T23:09:30.582 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2023-10-07T23:09:30.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:30 smithi123 ceph-mon[122926]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi086=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:30.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:30 smithi123 ceph-mon[122926]: Saving service mgr spec with placement smithi067=x;smithi086=y;smithi123=z;count:3 2023-10-07T23:09:30.659 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T23:09:30.659 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:30.659 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-07T23:09:30.660 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:30.660 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:30.660 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-07 23:08:37.699876289 +0000 2023-10-07T23:09:30.660 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-07 23:02:58.751314176 +0000 2023-10-07T23:09:30.660 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-07 23:02:58.751314176 +0000 2023-10-07T23:09:30.660 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-07T23:09:30.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T23:09:30.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:30 smithi086 ceph-mon[122131]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi086=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:30.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:30 smithi086 ceph-mon[122131]: Saving service mgr spec with placement smithi067=x;smithi086=y;smithi123=z;count:3 2023-10-07T23:09:30.751 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:30 smithi067 ceph-mon[82723]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi086=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:30.752 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:30 smithi067 ceph-mon[82723]: Saving service mgr spec with placement smithi067=x;smithi086=y;smithi123=z;count:3 2023-10-07T23:09:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-07T23:09:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-07T23:09:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000305003 s, 1.7 MB/s 2023-10-07T23:09:30.760 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T23:09:30.799 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2023-10-07T23:09:30.876 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T23:09:30.876 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:30.876 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-07T23:09:30.876 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:30.877 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:30.877 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-07 23:08:37.755875081 +0000 2023-10-07T23:09:30.877 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-07 23:02:59.548297022 +0000 2023-10-07T23:09:30.877 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-07 23:02:59.548297022 +0000 2023-10-07T23:09:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-07T23:09:30.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T23:09:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-07T23:09:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-07T23:09:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000298336 s, 1.7 MB/s 2023-10-07T23:09:30.978 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T23:09:31.016 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-07 23:08:37.816873766 +0000 2023-10-07T23:09:31.094 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-07 23:03:00.366279414 +0000 2023-10-07T23:09:31.095 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-07 23:03:00.366279414 +0000 2023-10-07T23:09:31.095 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-07T23:09:31.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T23:09:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-07T23:09:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-07T23:09:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000299604 s, 1.7 MB/s 2023-10-07T23:09:31.197 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T23:09:31.235 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-07T23:09:31.235 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T23:09:31.253 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T23:09:31.253 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 223544 Links: 1 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-07 23:09:16.348925893 +0000 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-07 22:59:50.136161888 +0000 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-07 22:59:50.136161888 +0000 2023-10-07T23:09:31.311 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-07T23:09:31.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T23:09:31.376 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-07T23:09:31.376 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-07T23:09:31.377 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000224445 s, 2.3 MB/s 2023-10-07T23:09:31.378 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T23:09:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-07T23:09:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:31.434 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2023-10-07T23:09:31.490 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T23:09:31.490 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:31.490 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 219997 Links: 1 2023-10-07T23:09:31.491 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:31.491 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:31.491 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-07 23:09:16.348925893 +0000 2023-10-07T23:09:31.491 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-07 22:59:50.641151120 +0000 2023-10-07T23:09:31.491 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-07 22:59:50.641151120 +0000 2023-10-07T23:09:31.491 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-07T23:09:31.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T23:09:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-07T23:09:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:31.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:31.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:31.553 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-07T23:09:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-07T23:09:31.555 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000275262 s, 1.9 MB/s 2023-10-07T23:09:31.556 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T23:09:31.611 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2023-10-07T23:09:31.667 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T23:09:31.667 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:31.667 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 224811 Links: 1 2023-10-07T23:09:31.667 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:31.667 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:31.667 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-07 23:09:16.349925871 +0000 2023-10-07T23:09:31.668 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-07 22:59:51.158140096 +0000 2023-10-07T23:09:31.668 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-07 22:59:51.158140096 +0000 2023-10-07T23:09:31.668 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-07T23:09:31.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T23:09:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-07T23:09:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-07T23:09:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00021959 s, 2.3 MB/s 2023-10-07T23:09:31.734 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T23:09:31.790 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2023-10-07T23:09:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T23:09:31.845 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:31.845 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 224953 Links: 1 2023-10-07T23:09:31.845 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:31.845 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:31.845 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-07 23:09:16.349925871 +0000 2023-10-07T23:09:31.846 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-07 22:59:51.658129433 +0000 2023-10-07T23:09:31.846 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-07 22:59:51.658129433 +0000 2023-10-07T23:09:31.846 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-07T23:09:31.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T23:09:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-07T23:09:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-07T23:09:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000221128 s, 2.3 MB/s 2023-10-07T23:09:31.909 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T23:09:31.965 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-07T23:09:31.965 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T23:09:31.980 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T23:09:31.980 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 224396 Links: 1 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-07 23:09:31.993169533 +0000 2023-10-07T23:09:32.036 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-07 22:59:49.499050949 +0000 2023-10-07T23:09:32.037 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-07 22:59:49.499050949 +0000 2023-10-07T23:09:32.037 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-07T23:09:32.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T23:09:32.166 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-07T23:09:32.167 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-07T23:09:32.167 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000243035 s, 2.1 MB/s 2023-10-07T23:09:32.168 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T23:09:32.183 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-10-07T23:09:32.238 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 224485 Links: 1 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-07 23:09:32.118166609 +0000 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-07 22:59:50.019038738 +0000 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-07 22:59:50.019038738 +0000 2023-10-07T23:09:32.239 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-07T23:09:32.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T23:09:32.306 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-07T23:09:32.307 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-07T23:09:32.307 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000223744 s, 2.3 MB/s 2023-10-07T23:09:32.308 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T23:09:32.364 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-10-07T23:09:32.418 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 223786 Links: 1 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-07 23:09:32.118166609 +0000 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-07 22:59:50.526026833 +0000 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-07 22:59:50.526026833 +0000 2023-10-07T23:09:32.419 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-07T23:09:32.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T23:09:32.479 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:32 smithi123 ceph-mon[122926]: Deploying daemon mgr.z on smithi123 2023-10-07T23:09:32.480 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:32 smithi123 systemd[1]: Starting Ceph mgr.z for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:09:32.481 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-07T23:09:32.482 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-07T23:09:32.482 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000224587 s, 2.3 MB/s 2023-10-07T23:09:32.483 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T23:09:32.539 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 217007 Links: 1 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-07 23:09:32.118166609 +0000 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-07 22:59:51.033014929 +0000 2023-10-07T23:09:32.596 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-07 22:59:51.033014929 +0000 2023-10-07T23:09:32.597 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-07T23:09:32.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T23:09:32.659 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-07T23:09:32.659 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-07T23:09:32.659 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000218755 s, 2.3 MB/s 2023-10-07T23:09:32.660 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T23:09:32.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:32 smithi086 ceph-mon[122131]: Deploying daemon mgr.z on smithi123 2023-10-07T23:09:32.715 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2023-10-07T23:09:32.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-07T23:09:32.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:32 smithi067 ceph-mon[82723]: Deploying daemon mgr.z on smithi123 2023-10-07T23:09:33.140 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:32 smithi123 ceph-mgr[125817]: -- 172.21.15.123:0/2909536725 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56205ee8c9c0 con 0x56205ee62800 2023-10-07T23:09:33.140 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:32 smithi123 bash[125635]: 8224dbad763778bb25f14f81a20c093b81f28031add736d4ea6814fb28592bc9 2023-10-07T23:09:33.141 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:32 smithi123 systemd[1]: Started Ceph mgr.z for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:09:33.497 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:09:33.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:33.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:33.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-07T23:09:33.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:33.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:33.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:33.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:33.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:33.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:33.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-07T23:09:33.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:33.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:34.308 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:33 smithi123 conmon[125743]: 2023-10-07T23:09:33.933+0000 7f4f5fe361c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:09:34.308 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:34 smithi123 conmon[125743]: 2023-10-07T23:09:34.086+0000 7f4f5fe361c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:09:34.309 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:34 smithi123 conmon[125743]: 2023-10-07T23:09:34.172+0000 7f4f5fe361c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:09:34.479 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:34 smithi067 ceph-mon[82723]: Deploying daemon mgr.y on smithi086 2023-10-07T23:09:34.578 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:34 smithi123 conmon[125743]: 2023-10-07T23:09:34.382+0000 7f4f5fe361c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:09:34.578 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:34 smithi123 ceph-mon[122926]: Deploying daemon mgr.y on smithi086 2023-10-07T23:09:34.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:34 smithi086 ceph-mon[122131]: Deploying daemon mgr.y on smithi086 2023-10-07T23:09:34.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:34 smithi086 systemd[1]: Starting Ceph mgr.y for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:09:34.890 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:34 smithi123 conmon[125743]: 2023-10-07T23:09:34.577+0000 7f4f5fe361c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:09:34.890 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:34 smithi123 conmon[125743]: 2023-10-07T23:09:34.732+0000 7f4f5fe361c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:09:34.890 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:34 smithi123 conmon[125743]: 2023-10-07T23:09:34.810+0000 7f4f5fe361c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:09:35.113 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:35 smithi086 ceph-mgr[123828]: -- 172.21.15.86:0/3387763730 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564899ede9c0 con 0x564899eb4800 2023-10-07T23:09:35.113 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:35 smithi086 bash[123763]: 670d3a75a35634b626bacb6e3e81083ccfe7ccb29fbc926c2efd95109f952ac7 2023-10-07T23:09:35.447 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:35 smithi086 systemd[1]: Started Ceph mgr.y for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:09:35.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:35 smithi086 ceph-mon[122131]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:35.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:35 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:35 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:35 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:35 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-07T23:09:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-07T23:09:35.587 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2023-10-07T23:09:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2023-10-07T23:09:35.587 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0294918 s, 356 MB/s 2023-10-07T23:09:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2023-10-07T23:09:35.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:35 smithi067 ceph-mon[82723]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:35.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:35 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:35 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:35 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:35 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.640 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:35 smithi123 conmon[125743]: 2023-10-07T23:09:35.382+0000 7f4f5fe361c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:09:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:35 smithi123 ceph-mon[122926]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:35 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:35 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:35 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:35 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:36.140 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:35 smithi123 conmon[125743]: 2023-10-07T23:09:35.890+0000 7f4f5fe361c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:09:36.140 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:35 smithi123 conmon[125743]: 2023-10-07T23:09:35.987+0000 7f4f5fe361c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:09:36.140 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:36 smithi123 conmon[125743]: 2023-10-07T23:09:36.086+0000 7f4f5fe361c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:09:36.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2023-10-07T23:09:36.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:36.573 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:36.573 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:36 smithi086 conmon[123803]: 2023-10-07T23:09:36.306+0000 7fa25f3261c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:09:36.573 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:36 smithi086 conmon[123803]: 2023-10-07T23:09:36.480+0000 7fa25f3261c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:09:36.573 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:36 smithi086 conmon[123803]: 2023-10-07T23:09:36.571+0000 7fa25f3261c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:09:36.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:36.947 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:36 smithi086 conmon[123803]: 2023-10-07T23:09:36.788+0000 7fa25f3261c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:09:36.975 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:09:36.977 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:36 smithi123 conmon[125743]: 2023-10-07T23:09:36.698+0000 7f4f5fe361c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T23:09:36.977 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:36 smithi123 conmon[125743]: 2023-10-07T23:09:36.882+0000 7f4f5fe361c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:09:37.318 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:36 smithi123 conmon[125743]: 2023-10-07T23:09:36.977+0000 7f4f5fe361c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:09:37.319 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:37 smithi123 conmon[125743]: 2023-10-07T23:09:37.105+0000 7f4f5fe361c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T23:09:37.319 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:37 smithi123 conmon[125743]: 2023-10-07T23:09:37.195+0000 7f4f5fe361c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:09:37.319 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:36 smithi086 conmon[123803]: 2023-10-07T23:09:36.997+0000 7fa25f3261c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:09:37.319 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:37 smithi086 conmon[123803]: 2023-10-07T23:09:37.155+0000 7fa25f3261c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:09:37.319 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:37 smithi086 conmon[123803]: 2023-10-07T23:09:37.235+0000 7fa25f3261c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:09:37.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:37.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-07T23:09:37.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:38.197 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:37 smithi086 conmon[123803]: 2023-10-07T23:09:37.804+0000 7fa25f3261c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:09:38.323 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.058+0000 7f4f5fe361c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T23:09:38.323 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.143+0000 7f4f5fe361c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:09:38.323 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.231+0000 7f4f5fe361c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 -- 172.21.15.67:0/4217367747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 msgr2=0x7efec80fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 --2- 172.21.15.67:0/4217367747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec80fd3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efebc0049c0 tx=0x7efebc009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 -- 172.21.15.67:0/4217367747 shutdown_connections 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 --2- 172.21.15.67:0/4217367747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fd8f0 0x7efec81041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 --2- 172.21.15.67:0/4217367747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec80fbd50 0x7efec80fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 --2- 172.21.15.67:0/4217367747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec80fd3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 -- 172.21.15.67:0/4217367747 >> 172.21.15.67:0/4217367747 conn(0x7efec80f74d0 msgr2=0x7efec80f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 -- 172.21.15.67:0/4217367747 shutdown_connections 2023-10-07T23:09:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.457+0000 7efece75d700 1 -- 172.21.15.67:0/4217367747 wait complete. 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.458+0000 7efece75d700 1 Processor -- start 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.458+0000 7efece75d700 1 -- start start 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efece75d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fbd50 0x7efec810fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efece75d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec8110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efece75d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec80fd8f0 0x7efec8117390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efece75d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec81179a0 con 0x7efec80fbd50 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efece75d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8117b10 con 0x7efec80fcf60 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efece75d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8117cd0 con 0x7efec80fd8f0 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fbd50 0x7efec810fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efecccfa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec80fd8f0 0x7efec8117390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec8110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fbd50 0x7efec810fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53860/0 (socket says 172.21.15.67:53860) 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec7fff700 1 -- 172.21.15.67:0/421118147 learned_addr learned my addr 172.21.15.67:0/421118147 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec77fe700 1 -- 172.21.15.67:0/421118147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec80fd8f0 msgr2=0x7efec8117390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec77fe700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec80fd8f0 0x7efec8117390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec77fe700 1 -- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fbd50 msgr2=0x7efec810fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec77fe700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fbd50 0x7efec810fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.459+0000 7efec77fe700 1 -- 172.21.15.67:0/421118147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc005040 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.460+0000 7efec77fe700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec8110300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efebc0049c0 tx=0x7efebc009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.460+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00e070 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.460+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efebc004160 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.460+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0123c0 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.460+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8117f50 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.460+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec8118300 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.461+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 43321+0+0 (secure 0 0 0) 0x7efebc009500 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.462+0000 7efec4ff9700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efeb00368e0 0x7efeb0038d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.462+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec8107590 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.462+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7efebc0497d0 con 0x7efec80fcf60 2023-10-07T23:09:38.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.462+0000 7efec7fff700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efeb00368e0 0x7efeb0038d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:09:38.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.462+0000 7efec7fff700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efeb00368e0 0x7efeb0038d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efeb8004ac0 tx=0x7efeb800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:09:38.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.467+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7efebc01c5f0 con 0x7efec80fcf60 2023-10-07T23:09:38.575 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:38 smithi067 ceph-mon[82723]: Reconfiguring mgr.x (unknown last config time)... 2023-10-07T23:09:38.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:38 smithi067 ceph-mon[82723]: Reconfiguring daemon mgr.x on smithi067 2023-10-07T23:09:38.609 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:38.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:38 smithi123 ceph-mon[122926]: Reconfiguring mgr.x (unknown last config time)... 2023-10-07T23:09:38.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:38 smithi123 ceph-mon[122926]: Reconfiguring daemon mgr.x on smithi067 2023-10-07T23:09:38.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:38.641 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.377+0000 7f4f5fe361c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:09:38.641 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.474+0000 7f4f5fe361c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:09:38.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:38 smithi086 conmon[123803]: 2023-10-07T23:09:38.277+0000 7fa25f3261c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:09:38.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:38 smithi086 conmon[123803]: 2023-10-07T23:09:38.368+0000 7fa25f3261c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:09:38.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:38 smithi086 conmon[123803]: 2023-10-07T23:09:38.457+0000 7fa25f3261c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:09:38.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:38 smithi086 ceph-mon[122131]: Reconfiguring mgr.x (unknown last config time)... 2023-10-07T23:09:38.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:38 smithi086 ceph-mon[122131]: Reconfiguring daemon mgr.x on smithi067 2023-10-07T23:09:38.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:38.697+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efec81184f0 con 0x7efeb00368e0 2023-10-07T23:09:39.121 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.856+0000 7f4f5fe361c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:09:39.121 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:38 smithi123 conmon[125743]: 2023-10-07T23:09:38.945+0000 7f4f5fe361c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:09:39.121 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:39 smithi123 conmon[125743]: 2023-10-07T23:09:39.031+0000 7f4f5fe361c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:09:39.286 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:39 smithi086 conmon[123803]: 2023-10-07T23:09:39.032+0000 7fa25f3261c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-07T23:09:39.286 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:39 smithi086 conmon[123803]: 2023-10-07T23:09:39.198+0000 7fa25f3261c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:09:39.286 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:39 smithi086 conmon[123803]: 2023-10-07T23:09:39.286+0000 7fa25f3261c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:09:39.390 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:09:39 smithi123 conmon[125743]: 2023-10-07T23:09:39.121+0000 7f4f5fe361c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:39.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: Standby manager daemon z started 2023-10-07T23:09:39.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-07T23:09:39.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-07T23:09:39.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-07T23:09:39.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:39 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-07T23:09:39.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:39 smithi086 conmon[123803]: 2023-10-07T23:09:39.412+0000 7fa25f3261c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-07T23:09:39.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:39 smithi086 conmon[123803]: 2023-10-07T23:09:39.499+0000 7fa25f3261c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:09:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:09:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:09:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: Standby manager daemon z started 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-07T23:09:39.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-07T23:09:39.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:39 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-07T23:09:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:09:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: Standby manager daemon z started 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-07T23:09:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:39 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.123:0/573508491' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-07T23:09:39.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:39.956+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) v1 ==== 86302+0+0 (secure 0 0 0) 0x7efebc012810 con 0x7efec80fcf60 2023-10-07T23:09:40.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:40 smithi067 ceph-mon[82723]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:40.613 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:40 smithi067 ceph-mon[82723]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-07T23:09:40.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:40 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-07T23:09:40.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:40 smithi123 ceph-mon[122926]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:40.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:40 smithi123 ceph-mon[122926]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-07T23:09:40.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:40 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-07T23:09:40.662 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:40 smithi086 ceph-mon[122131]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:09:40.663 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:40 smithi086 ceph-mon[122131]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-07T23:09:40.663 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:40 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-07T23:09:40.663 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:40 smithi086 conmon[123803]: 2023-10-07T23:09:40.348+0000 7fa25f3261c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-07T23:09:40.663 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:40 smithi086 conmon[123803]: 2023-10-07T23:09:40.433+0000 7fa25f3261c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:09:40.663 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:40 smithi086 conmon[123803]: 2023-10-07T23:09:40.519+0000 7fa25f3261c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:09:40.663 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:40 smithi086 conmon[123803]: 2023-10-07T23:09:40.662+0000 7fa25f3261c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:09:40.947 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:40 smithi086 conmon[123803]: 2023-10-07T23:09:40.757+0000 7fa25f3261c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:09:41.401 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:41 smithi086 ceph-mon[122131]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:41.401 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:41 smithi086 conmon[123803]: 2023-10-07T23:09:41.140+0000 7fa25f3261c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:09:41.401 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:41 smithi086 conmon[123803]: 2023-10-07T23:09:41.227+0000 7fa25f3261c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:09:41.401 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:41 smithi086 conmon[123803]: 2023-10-07T23:09:41.313+0000 7fa25f3261c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:09:41.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:41 smithi123 ceph-mon[122926]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:41.697 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:09:41 smithi086 conmon[123803]: 2023-10-07T23:09:41.401+0000 7fa25f3261c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:09:41.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:41 smithi067 ceph-mon[82723]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:42.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:42.343+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mon.2 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 15) v1 ==== 129283+0+0 (secure 0 0 0) 0x7efebc028100 con 0x7efec80fcf60 2023-10-07T23:09:42.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: Standby manager daemon y started 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/545109551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]': finished 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: osdmap e5: 1 total, 0 up, 1 in 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:09:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:42 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2151253992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:09:42.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: Standby manager daemon y started 2023-10-07T23:09:42.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-07T23:09:42.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-07T23:09:42.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/545109551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]: dispatch 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]: dispatch 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]': finished 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: osdmap e5: 1 total, 0 up, 1 in 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:09:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:42 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2151253992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:09:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: Standby manager daemon y started 2023-10-07T23:09:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-07T23:09:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-07T23:09:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-07T23:09:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='mgr.? 172.21.15.86:0/988876682' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-07T23:09:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/545109551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]: dispatch 2023-10-07T23:09:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]: dispatch 2023-10-07T23:09:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7d0d95f-5162-4fdd-8f39-a33aa62127a6"}]': finished 2023-10-07T23:09:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: osdmap e5: 1 total, 0 up, 1 in 2023-10-07T23:09:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:09:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:42 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2151253992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:09:43.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:43 smithi123 ceph-mon[122926]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:43.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:43 smithi123 ceph-mon[122926]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-07T23:09:43.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:43 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-07T23:09:43.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:43 smithi086 ceph-mon[122131]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:43.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:43 smithi086 ceph-mon[122131]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-07T23:09:43.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:43 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-07T23:09:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:43 smithi067 ceph-mon[82723]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:43 smithi067 ceph-mon[82723]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-07T23:09:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:43 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-07T23:09:45.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:44 smithi067 ceph-mon[82723]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:45.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:44 smithi123 ceph-mon[122926]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:45.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:44 smithi086 ceph-mon[122131]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:47.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:46 smithi067 ceph-mon[82723]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:46 smithi086 ceph-mon[122131]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:47.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:46 smithi123 ceph-mon[122926]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:48 smithi086 ceph-mon[122131]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:48 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-07T23:09:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:48 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:48 smithi067 ceph-mon[82723]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-07T23:09:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:48 smithi123 ceph-mon[122926]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:48 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-07T23:09:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:48 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:50.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:49 smithi067 ceph-mon[82723]: Deploying daemon osd.0 on smithi067 2023-10-07T23:09:50.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:49 smithi086 ceph-mon[122131]: Deploying daemon osd.0 on smithi067 2023-10-07T23:09:50.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:49 smithi123 ceph-mon[122926]: Deploying daemon osd.0 on smithi067 2023-10-07T23:09:51.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:50 smithi086 ceph-mon[122131]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:50 smithi067 ceph-mon[82723]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:50 smithi123 ceph-mon[122926]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:53.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:52 smithi086 ceph-mon[122131]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:52 smithi067 ceph-mon[82723]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:53.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:52 smithi123 ceph-mon[122926]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:54.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:54 smithi067 ceph-mon[82723]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:55.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:54 smithi123 ceph-mon[122926]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:55.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:54 smithi086 ceph-mon[122131]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:56.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:56.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:56.493 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:56.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:56.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:56.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:56.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:56.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:56.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:57.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:57 smithi067 ceph-mon[82723]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:57.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:57 smithi123 ceph-mon[122926]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:57.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:57 smithi086 ceph-mon[122131]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:58.453 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2023-10-07T23:09:58.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.453+0000 7efec4ff9700 1 -- 172.21.15.67:0/421118147 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efec81184f0 con 0x7efeb00368e0 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efeb00368e0 msgr2=0x7efeb0038d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efeb00368e0 0x7efeb0038d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efeb8004ac0 tx=0x7efeb800b040 comp rx=0 tx=0).stop 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 msgr2=0x7efec8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec8110300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efebc0049c0 tx=0x7efebc009e30 comp rx=0 tx=0).stop 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 shutdown_connections 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efeb00368e0 0x7efeb0038d90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efec80fd8f0 0x7efec8117390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:58.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efec80fbd50 0x7efec810fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 --2- 172.21.15.67:0/421118147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec80fcf60 0x7efec8110300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:09:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 >> 172.21.15.67:0/421118147 conn(0x7efec80f74d0 msgr2=0x7efec8065650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:09:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 shutdown_connections 2023-10-07T23:09:58.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:09:58.455+0000 7efece75d700 1 -- 172.21.15.67:0/421118147 wait complete. 2023-10-07T23:09:58.883 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-07T23:09:58.888 INFO:tasks.cephadm:Deploying osd.1 on smithi086 with /dev/vg_nvme/lv_4... 2023-10-07T23:09:58.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-07T23:09:58.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:58.917 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:58.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:58.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.918 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:58.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:58.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:58.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:58.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:58.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:58.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:09:59.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:09:59.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:09:59.473 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:09:59.504 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:09:59 smithi067 conmon[101320]: 2023-10-07T23:09:59.216+0000 7fcee0b14540 -1 osd.0 0 log_to_monitors true 2023-10-07T23:09:59.889 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:59 smithi086 ceph-mon[122131]: from='osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-07T23:09:59.889 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:09:59 smithi086 ceph-mon[122131]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-07T23:10:00.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:59 smithi067 ceph-mon[82723]: from='osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-07T23:10:00.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:09:59 smithi067 ceph-mon[82723]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-07T23:10:00.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:59 smithi123 ceph-mon[122926]: from='osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-07T23:10:00.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:09:59 smithi123 ceph-mon[122926]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-07T23:10:00.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-07T23:10:00.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: osdmap e6: 1 total, 0 up, 1 in 2023-10-07T23:10:00.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:00.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: from='osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:10:00.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:10:00.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:10:00.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:00 smithi086 ceph-mon[122131]: overall HEALTH_OK 2023-10-07T23:10:01.004 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:00 smithi067 conmon[101320]: 2023-10-07T23:10:00.756+0000 7fced72f9700 -1 osd.0 0 waiting for initial osdmap 2023-10-07T23:10:01.005 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:00 smithi067 conmon[101320]: 2023-10-07T23:10:00.762+0000 7fced08f1700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: osdmap e6: 1 total, 0 up, 1 in 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: from='osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:10:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:00 smithi067 ceph-mon[82723]: overall HEALTH_OK 2023-10-07T23:10:01.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-07T23:10:01.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: osdmap e6: 1 total, 0 up, 1 in 2023-10-07T23:10:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: from='osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:10:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:10:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:10:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:00 smithi123 ceph-mon[122926]: overall HEALTH_OK 2023-10-07T23:10:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-07T23:10:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-07T23:10:01.187 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2023-10-07T23:10:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2023-10-07T23:10:01.187 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010962 s, 957 MB/s 2023-10-07T23:10:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2023-10-07T23:10:01.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2023-10-07T23:10:02.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:01 smithi067 ceph-mon[82723]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:10:02.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:01 smithi067 ceph-mon[82723]: osdmap e7: 1 total, 0 up, 1 in 2023-10-07T23:10:02.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:01 smithi123 ceph-mon[122926]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:10:02.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:01 smithi123 ceph-mon[122926]: osdmap e7: 1 total, 0 up, 1 in 2023-10-07T23:10:02.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:01 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:01 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:01 smithi086 ceph-mon[122131]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:10:02.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:01 smithi086 ceph-mon[122131]: osdmap e7: 1 total, 0 up, 1 in 2023-10-07T23:10:02.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:01 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:01 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:10:02.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:10:02.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:10:02.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] boot 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: osdmap e8: 1 total, 1 up, 1 in 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] boot 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: osdmap e8: 1 total, 1 up, 1 in 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:03.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:03.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:03.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: osd.0 [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] boot 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: osdmap e8: 1 total, 1 up, 1 in 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:03.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:03.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 -- 172.21.15.86:0/316653052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 msgr2=0x7f7d080fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/316653052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d080fc160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0049c0 tx=0x7f7cfc009e30 comp rx=0 tx=0).stop 2023-10-07T23:10:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 -- 172.21.15.86:0/316653052 shutdown_connections 2023-10-07T23:10:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/316653052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d080fc160 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/316653052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080fd8f0 0x7f7d081041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/316653052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d080fcf60 0x7f7d080fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:03.838 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 -- 172.21.15.86:0/316653052 >> 172.21.15.86:0/316653052 conn(0x7f7d080f74b0 msgr2=0x7f7d080f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:10:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 -- 172.21.15.86:0/316653052 shutdown_connections 2023-10-07T23:10:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.836+0000 7f7d0efb1700 1 -- 172.21.15.86:0/316653052 wait complete. 2023-10-07T23:10:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 Processor -- start 2023-10-07T23:10:03.839 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 -- start start 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d08066c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d080fcf60 0x7f7d08112b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080fd8f0 0x7f7d081151a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08115990 con 0x7f7d080fcf60 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08115b00 con 0x7f7d080fbd50 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0efb1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08115cc0 con 0x7f7d080fd8f0 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d08066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d08066c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47684/0 (socket says 172.21.15.86:47684) 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 -- 172.21.15.86:0/3917970939 learned_addr learned my addr 172.21.15.86:0/3917970939 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-07T23:10:03.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0d54e700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080fd8f0 0x7f7d081151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d07fff700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d080fcf60 0x7f7d08112b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 -- 172.21.15.86:0/3917970939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080fd8f0 msgr2=0x7f7d081151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080fd8f0 0x7f7d081151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 -- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d080fcf60 msgr2=0x7f7d08112b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d080fcf60 0x7f7d08112b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.837+0000 7f7d0cd4d700 1 -- 172.21.15.86:0/3917970939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc005040 con 0x7f7d080fbd50 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.838+0000 7f7d0cd4d700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d08066c00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00bfa0 tx=0x7f7cfc009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.838+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00e040 con 0x7f7d080fbd50 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.838+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d08115ee0 con 0x7f7d080fbd50 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.838+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cfc0096f0 con 0x7f7d080fbd50 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.838+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc013df0 con 0x7f7d080fbd50 2023-10-07T23:10:03.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.838+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d08116280 con 0x7f7d080fbd50 2023-10-07T23:10:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.839+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d081033f0 con 0x7f7d080fbd50 2023-10-07T23:10:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.840+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) v1 ==== 129283+0+0 (secure 0 0 0) 0x7f7cfc009220 con 0x7f7d080fbd50 2023-10-07T23:10:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.840+0000 7f7d057fa700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7cf809b6e0 0x7f7cf809db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.840+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1763+0+0 (secure 0 0 0) 0x7f7cfc0c46f0 con 0x7f7d080fbd50 2023-10-07T23:10:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.840+0000 7f7d07fff700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7cf809b6e0 0x7f7cf809db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:03.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.841+0000 7f7d07fff700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7cf809b6e0 0x7f7cf809db90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4006fd0 tx=0x7f7cf4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:10:03.845 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:03.844+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7cfc091970 con 0x7f7d080fbd50 2023-10-07T23:10:03.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:03 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi067 2023-10-07T23:10:03.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:03 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi067 to 17267M 2023-10-07T23:10:03.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:03 smithi086 ceph-mon[122131]: osdmap e9: 1 total, 1 up, 1 in 2023-10-07T23:10:04.038 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:04.037+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d0802b450 con 0x7f7cf809b6e0 2023-10-07T23:10:04.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:03 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi067 2023-10-07T23:10:04.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:03 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi067 to 17267M 2023-10-07T23:10:04.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:03 smithi123 ceph-mon[122926]: osdmap e9: 1 total, 1 up, 1 in 2023-10-07T23:10:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:03 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi067 2023-10-07T23:10:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:03 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi067 to 17267M 2023-10-07T23:10:04.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:03 smithi067 ceph-mon[82723]: osdmap e9: 1 total, 1 up, 1 in 2023-10-07T23:10:05.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:04 smithi123 ceph-mon[122926]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:05.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:04 smithi123 ceph-mon[122926]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:05.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:05.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:05.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:05.159 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:04 smithi086 ceph-mon[122131]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:05.160 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:04 smithi086 ceph-mon[122131]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:05.160 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:05.160 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:05.160 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:04 smithi067 ceph-mon[82723]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:04 smithi067 ceph-mon[82723]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:06 smithi086 ceph-mon[122131]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:06 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/3912248758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]: dispatch 2023-10-07T23:10:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:06 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]: dispatch 2023-10-07T23:10:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:06 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]': finished 2023-10-07T23:10:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:06 smithi086 ceph-mon[122131]: osdmap e10: 2 total, 1 up, 2 in 2023-10-07T23:10:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:06 smithi067 ceph-mon[82723]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:06 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/3912248758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]: dispatch 2023-10-07T23:10:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:06 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]: dispatch 2023-10-07T23:10:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:06 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]': finished 2023-10-07T23:10:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:06 smithi067 ceph-mon[82723]: osdmap e10: 2 total, 1 up, 2 in 2023-10-07T23:10:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:06 smithi123 ceph-mon[122926]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:06 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/3912248758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]: dispatch 2023-10-07T23:10:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:06 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]: dispatch 2023-10-07T23:10:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:06 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04b0f276-f836-435c-88ac-d6832505fc36"}]': finished 2023-10-07T23:10:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:06 smithi123 ceph-mon[122926]: osdmap e10: 2 total, 1 up, 2 in 2023-10-07T23:10:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:08.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:07 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/3139343765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:08.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:07 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/3139343765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:08.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:07 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/3139343765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:09.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:08 smithi086 ceph-mon[122131]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:09.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:08 smithi067 ceph-mon[82723]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:09.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:08 smithi123 ceph-mon[122926]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:11.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:10 smithi086 ceph-mon[122131]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:10 smithi067 ceph-mon[82723]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:11.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:10 smithi123 ceph-mon[122926]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:13.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:12 smithi086 ceph-mon[122131]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:13.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:12 smithi067 ceph-mon[82723]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:12 smithi123 ceph-mon[122926]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:15.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:14 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-07T23:10:15.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:14 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:15.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:14 smithi067 ceph-mon[82723]: Deploying daemon osd.1 on smithi086 2023-10-07T23:10:15.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:14 smithi067 ceph-mon[82723]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:15.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:14 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-07T23:10:15.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:14 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:14 smithi123 ceph-mon[122926]: Deploying daemon osd.1 on smithi086 2023-10-07T23:10:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:14 smithi123 ceph-mon[122926]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:14 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-07T23:10:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:14 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:14 smithi086 ceph-mon[122131]: Deploying daemon osd.1 on smithi086 2023-10-07T23:10:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:14 smithi086 ceph-mon[122131]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:17.075 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:16 smithi086 ceph-mon[122131]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:17.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:16 smithi067 ceph-mon[82723]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:17.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:16 smithi123 ceph-mon[122926]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:19.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:18 smithi086 ceph-mon[122131]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:18 smithi067 ceph-mon[82723]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:18 smithi123 ceph-mon[122926]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:19.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:21.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:20 smithi086 ceph-mon[122131]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:21.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:20 smithi067 ceph-mon[82723]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:20 smithi123 ceph-mon[122926]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:22.160 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.160+0000 7f7d057fa700 1 -- 172.21.15.86:0/3917970939 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d0802b450 con 0x7f7cf809b6e0 2023-10-07T23:10:22.161 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 1 on host 'smithi086' 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7cf809b6e0 msgr2=0x7f7cf809db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7cf809b6e0 0x7f7cf809db90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4006fd0 tx=0x7f7cf4008040 comp rx=0 tx=0).stop 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 msgr2=0x7f7d08066c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d08066c00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00bfa0 tx=0x7f7cfc009e30 comp rx=0 tx=0).stop 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 shutdown_connections 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d080fbd50 0x7f7d08066c00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7cf809b6e0 0x7f7cf809db90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:22.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d080fd8f0 0x7f7d081151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:22.164 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 --2- 172.21.15.86:0/3917970939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d080fcf60 0x7f7d08112b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:22.164 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.161+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 >> 172.21.15.86:0/3917970939 conn(0x7f7d080f74b0 msgr2=0x7f7d08100570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:10:22.164 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.162+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 shutdown_connections 2023-10-07T23:10:22.164 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:10:22.162+0000 7f7d0efb1700 1 -- 172.21.15.86:0/3917970939 wait complete. 2023-10-07T23:10:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:22.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:22.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:22.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:22.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:22.874 DEBUG:teuthology.orchestra.run.smithi086:osd.1> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-07T23:10:22.876 INFO:tasks.cephadm:Deploying osd.2 on smithi123 with /dev/vg_nvme/lv_4... 2023-10-07T23:10:22.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-07T23:10:22.905 INFO:journalctl@ceph.osd.1.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:10:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:23 smithi123 ceph-mon[122926]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:23.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:23.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:23.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:23.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:23 smithi123 ceph-mon[122926]: from='osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-07T23:10:23.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:23 smithi123 ceph-mon[122926]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-07T23:10:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:23 smithi086 ceph-mon[122131]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:23.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:23.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:23 smithi086 ceph-mon[122131]: from='osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-07T23:10:23.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:23 smithi086 ceph-mon[122131]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-07T23:10:23.460 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:10:23.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:23 smithi067 ceph-mon[82723]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:23.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:23 smithi067 ceph-mon[82723]: from='osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-07T23:10:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:23 smithi067 ceph-mon[82723]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-07T23:10:24.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:24 smithi086 ceph-mon[122131]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-07T23:10:24.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:24 smithi086 ceph-mon[122131]: osdmap e11: 2 total, 1 up, 2 in 2023-10-07T23:10:24.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:24 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:24.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:24 smithi086 ceph-mon[122131]: from='osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:10:24.375 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:24 smithi086 ceph-mon[122131]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:10:24.376 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:24 smithi086 conmon[128550]: 2023-10-07T23:10:24.162+0000 7f82e3fc7700 -1 osd.1 0 waiting for initial osdmap 2023-10-07T23:10:24.376 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:24 smithi086 conmon[128550]: 2023-10-07T23:10:24.166+0000 7f82dcdbe700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:10:24.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:24 smithi123 ceph-mon[122926]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-07T23:10:24.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:24 smithi123 ceph-mon[122926]: osdmap e11: 2 total, 1 up, 2 in 2023-10-07T23:10:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:24 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:24 smithi123 ceph-mon[122926]: from='osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:10:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:24 smithi123 ceph-mon[122926]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:10:24.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:24 smithi067 ceph-mon[82723]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-07T23:10:24.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:24 smithi067 ceph-mon[82723]: osdmap e11: 2 total, 1 up, 2 in 2023-10-07T23:10:24.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:24 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:24.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:24 smithi067 ceph-mon[82723]: from='osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:10:24.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:24 smithi067 ceph-mon[82723]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:10:25.244 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-07T23:10:25.244 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-07T23:10:25.244 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-10-07T23:10:25.244 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-10-07T23:10:25.244 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113463 s, 924 MB/s 2023-10-07T23:10:25.244 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-10-07T23:10:25.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:25 smithi086 ceph-mon[122131]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:25.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:25 smithi086 ceph-mon[122131]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:10:25.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:25 smithi086 ceph-mon[122131]: osdmap e12: 2 total, 1 up, 2 in 2023-10-07T23:10:25.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:25.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:25.473 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:25 smithi123 ceph-mon[122926]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:25.473 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:25 smithi123 ceph-mon[122926]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:10:25.473 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:25 smithi123 ceph-mon[122926]: osdmap e12: 2 total, 1 up, 2 in 2023-10-07T23:10:25.473 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:25.474 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:25 smithi067 ceph-mon[82723]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:25 smithi067 ceph-mon[82723]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:10:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:25 smithi067 ceph-mon[82723]: osdmap e12: 2 total, 1 up, 2 in 2023-10-07T23:10:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:25.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:25.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2023-10-07T23:10:26.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:10:26.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:10:26.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:26.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] boot 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: osdmap e13: 2 total, 2 up, 2 in 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:26.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.467 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:10:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:10:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] boot 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: osdmap e13: 2 total, 2 up, 2 in 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:26.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:10:26.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:10:26.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:26.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: osd.1 [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] boot 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: osdmap e13: 2 total, 2 up, 2 in 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:26.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:27 smithi123 ceph-mon[122926]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:27 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi086 2023-10-07T23:10:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:27 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi086 to 17172M 2023-10-07T23:10:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:27 smithi123 ceph-mon[122926]: osdmap e14: 2 total, 2 up, 2 in 2023-10-07T23:10:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:27 smithi086 ceph-mon[122131]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:27 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi086 2023-10-07T23:10:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:27 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi086 to 17172M 2023-10-07T23:10:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:27 smithi086 ceph-mon[122131]: osdmap e14: 2 total, 2 up, 2 in 2023-10-07T23:10:27.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:27 smithi067 ceph-mon[82723]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2023-10-07T23:10:27.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:27 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi086 2023-10-07T23:10:27.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:27 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi086 to 17172M 2023-10-07T23:10:27.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:27 smithi067 ceph-mon[82723]: osdmap e14: 2 total, 2 up, 2 in 2023-10-07T23:10:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 -- 172.21.15.123:0/2275580944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fbf90 msgr2=0x7fcf180fc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:27.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/2275580944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fbf90 0x7fcf180fc400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf080049f0 tx=0x7fcf08009e30 comp rx=0 tx=0).stop 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 -- 172.21.15.123:0/2275580944 shutdown_connections 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/2275580944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf180fc940 0x7fcf181041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/2275580944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf180fd320 0x7fcf180fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/2275580944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fbf90 0x7fcf180fc400 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.912+0000 7fcf1fb35700 1 -- 172.21.15.123:0/2275580944 >> 172.21.15.123:0/2275580944 conn(0x7fcf180f6c30 msgr2=0x7fcf180f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 -- 172.21.15.123:0/2275580944 shutdown_connections 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 -- 172.21.15.123:0/2275580944 wait complete. 2023-10-07T23:10:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 Processor -- start 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 -- start start 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf180fbf90 0x7fcf180fb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf180fc940 0x7fcf18112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 0x7fcf18114eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18115650 con 0x7fcf180fc940 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181157c0 con 0x7fcf180fbf90 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1fb35700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18115980 con 0x7fcf180fd320 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1e0d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 0x7fcf18114eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1e0d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 0x7fcf18114eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37648/0 (socket says 172.21.15.123:37648) 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.913+0000 7fcf1e0d2700 1 -- 172.21.15.123:0/1704078425 learned_addr learned my addr 172.21.15.123:0/1704078425 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1d8d1700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf180fbf90 0x7fcf180fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1e0d2700 1 -- 172.21.15.123:0/1704078425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf180fbf90 msgr2=0x7fcf180fb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1e0d2700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf180fbf90 0x7fcf180fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1e0d2700 1 -- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf180fc940 msgr2=0x7fcf18112870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1e0d2700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf180fc940 0x7fcf18112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1e0d2700 1 -- 172.21.15.123:0/1704078425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf08005040 con 0x7fcf180fd320 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1e0d2700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 0x7fcf18114eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c960 tx=0x7fcf10007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10004020 con 0x7fcf180fd320 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf18115c60 con 0x7fcf180fd320 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf10004180 con 0x7fcf180fd320 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10003920 con 0x7fcf180fd320 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.914+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf18116180 con 0x7fcf180fd320 2023-10-07T23:10:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.915+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18102bb0 con 0x7fcf180fd320 2023-10-07T23:10:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.916+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) v1 ==== 129283+0+0 (secure 0 0 0) 0x7fcf10003d50 con 0x7fcf180fd320 2023-10-07T23:10:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.916+0000 7fcf0e7fc700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf0409b6e0 0x7fcf0409db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.917+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2305+0+0 (secure 0 0 0) 0x7fcf100c3a00 con 0x7fcf180fd320 2023-10-07T23:10:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.917+0000 7fcf1d8d1700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf0409b6e0 0x7fcf0409db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.917+0000 7fcf1d8d1700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf0409b6e0 0x7fcf0409db90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf18065c00 tx=0x7fcf14009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:10:27.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:27.922+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fcf10090a60 con 0x7fcf180fd320 2023-10-07T23:10:28.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:28.110+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf1802b470 con 0x7fcf0409b6e0 2023-10-07T23:10:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:28.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:28.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:28.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:28.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:28.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:29.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:29 smithi123 ceph-mon[122926]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:29.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:29 smithi123 ceph-mon[122926]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:29 smithi086 ceph-mon[122131]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:29 smithi086 ceph-mon[122131]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:29.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:29 smithi067 ceph-mon[82723]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:29.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:29 smithi067 ceph-mon[82723]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:31 smithi086 ceph-mon[122131]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:31 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/1871991080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]: dispatch 2023-10-07T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:31 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]: dispatch 2023-10-07T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:31 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]': finished 2023-10-07T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:31 smithi086 ceph-mon[122131]: osdmap e15: 3 total, 2 up, 3 in 2023-10-07T23:10:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:31 smithi067 ceph-mon[82723]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:31.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:31 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/1871991080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]: dispatch 2023-10-07T23:10:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:31 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]: dispatch 2023-10-07T23:10:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:31 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]': finished 2023-10-07T23:10:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:31 smithi067 ceph-mon[82723]: osdmap e15: 3 total, 2 up, 3 in 2023-10-07T23:10:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:31.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:31 smithi123 ceph-mon[122926]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:31.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:31 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/1871991080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]: dispatch 2023-10-07T23:10:31.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:31 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]: dispatch 2023-10-07T23:10:31.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:31 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135"}]': finished 2023-10-07T23:10:31.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:31 smithi123 ceph-mon[122926]: osdmap e15: 3 total, 2 up, 3 in 2023-10-07T23:10:31.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:32.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/4210282222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:32.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/4210282222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:32.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/4210282222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:33.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:33 smithi086 ceph-mon[122131]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:33.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:33 smithi067 ceph-mon[82723]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:33.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:33 smithi123 ceph-mon[122926]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:35.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:34 smithi067 ceph-mon[82723]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:35.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:34 smithi123 ceph-mon[122926]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:35.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:34 smithi086 ceph-mon[122131]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:37.195 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:36 smithi123 ceph-mon[122926]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:37.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:36 smithi086 ceph-mon[122131]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:37.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:36 smithi067 ceph-mon[82723]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:39.167 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:38 smithi123 ceph-mon[122926]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:39.167 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-07T23:10:39.167 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:39.167 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:38 smithi123 ceph-mon[122926]: Deploying daemon osd.2 on smithi123 2023-10-07T23:10:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:38 smithi086 ceph-mon[122131]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-07T23:10:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:38 smithi086 ceph-mon[122131]: Deploying daemon osd.2 on smithi123 2023-10-07T23:10:39.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:38 smithi067 ceph-mon[82723]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:39.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-07T23:10:39.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:39.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:38 smithi067 ceph-mon[82723]: Deploying daemon osd.2 on smithi123 2023-10-07T23:10:41.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:40 smithi086 ceph-mon[122131]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:41.204 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:40 smithi123 ceph-mon[122926]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:41.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:40 smithi067 ceph-mon[82723]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:43.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:42 smithi086 ceph-mon[122131]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:43.214 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:42 smithi123 ceph-mon[122926]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:43.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:42 smithi067 ceph-mon[82723]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:44.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:44 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:44.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:44 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:44.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:44 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:44.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:44 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:44.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:44 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:44.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:44 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:44.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:44 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:44.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:44 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:44.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:44 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:45.528 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:45 smithi123 ceph-mon[122926]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:45.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:45 smithi086 ceph-mon[122131]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:45.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:45 smithi067 ceph-mon[82723]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:46.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.731+0000 7fcf0e7fc700 1 -- 172.21.15.123:0/1704078425 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcf1802b470 con 0x7fcf0409b6e0 2023-10-07T23:10:46.732 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 2 on host 'smithi123' 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.733+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf0409b6e0 msgr2=0x7fcf0409db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.733+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf0409b6e0 0x7fcf0409db90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf18065c00 tx=0x7fcf14009040 comp rx=0 tx=0).stop 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 msgr2=0x7fcf18114eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 0x7fcf18114eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c960 tx=0x7fcf10007ec0 comp rx=0 tx=0).stop 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 shutdown_connections 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf180fbf90 0x7fcf180fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fcf0409b6e0 0x7fcf0409db90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:46.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf180fd320 0x7fcf18114eb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:46.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 --2- 172.21.15.123:0/1704078425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf180fc940 0x7fcf18112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:46.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 >> 172.21.15.123:0/1704078425 conn(0x7fcf180f6c30 msgr2=0x7fcf180f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:10:46.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 shutdown_connections 2023-10-07T23:10:46.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:10:46.734+0000 7fcf1fb35700 1 -- 172.21.15.123:0/1704078425 wait complete. 2023-10-07T23:10:46.945 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:46.945 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:46.945 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:46.945 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:46.945 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:46.945 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:46.946 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:46.946 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:46.946 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:10:47.419 DEBUG:teuthology.orchestra.run.smithi123:osd.2> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-07T23:10:47.421 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_3... 2023-10-07T23:10:47.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-07T23:10:47.450 INFO:journalctl@ceph.osd.2.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:10:48.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:47 smithi123 ceph-mon[122926]: from='osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-07T23:10:48.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:47 smithi123 ceph-mon[122926]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-07T23:10:48.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:47 smithi086 ceph-mon[122131]: from='osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-07T23:10:48.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:47 smithi086 ceph-mon[122131]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-07T23:10:48.198 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:10:48.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:47 smithi067 ceph-mon[82723]: from='osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-07T23:10:48.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:47 smithi067 ceph-mon[82723]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-07T23:10:48.854 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:48 smithi123 conmon[130600]: 2023-10-07T23:10:48.791+0000 7fddea564700 -1 osd.2 0 waiting for initial osdmap 2023-10-07T23:10:48.854 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:48 smithi123 conmon[130600]: 2023-10-07T23:10:48.794+0000 7fdde7363700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:10:48.854 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:48 smithi123 ceph-mon[122926]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-07T23:10:48.855 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:48 smithi123 ceph-mon[122926]: osdmap e16: 3 total, 2 up, 3 in 2023-10-07T23:10:48.855 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:48 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:48.855 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:48 smithi123 ceph-mon[122926]: from='osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:10:48.855 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:48 smithi123 ceph-mon[122926]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:10:48.855 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:48 smithi123 ceph-mon[122926]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:48 smithi086 ceph-mon[122131]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-07T23:10:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:48 smithi086 ceph-mon[122131]: osdmap e16: 3 total, 2 up, 3 in 2023-10-07T23:10:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:48 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:49.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:48 smithi086 ceph-mon[122131]: from='osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:10:49.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:48 smithi086 ceph-mon[122131]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:10:49.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:48 smithi086 ceph-mon[122131]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:48 smithi067 ceph-mon[82723]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-07T23:10:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:48 smithi067 ceph-mon[82723]: osdmap e16: 3 total, 2 up, 3 in 2023-10-07T23:10:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:48 smithi067 ceph-mon[82723]: from='osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:10:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:48 smithi067 ceph-mon[82723]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:10:49.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:48 smithi067 ceph-mon[82723]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2023-10-07T23:10:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-07T23:10:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-07T23:10:50.086 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2023-10-07T23:10:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2023-10-07T23:10:50.086 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0306491 s, 342 MB/s 2023-10-07T23:10:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2023-10-07T23:10:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:49 smithi067 ceph-mon[82723]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:10:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:49 smithi067 ceph-mon[82723]: osdmap e17: 3 total, 2 up, 3 in 2023-10-07T23:10:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:50.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:49 smithi123 ceph-mon[122926]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:10:50.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:49 smithi123 ceph-mon[122926]: osdmap e17: 3 total, 2 up, 3 in 2023-10-07T23:10:50.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:50.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:50.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:49 smithi086 ceph-mon[122131]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:10:50.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:49 smithi086 ceph-mon[122131]: osdmap e17: 3 total, 2 up, 3 in 2023-10-07T23:10:50.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:50.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:50.665 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2023-10-07T23:10:51.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] boot 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: osdmap e18: 3 total, 3 up, 3 in 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: pgmap v79: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi123 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:51.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:51.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:51.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:50 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] boot 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: osdmap e18: 3 total, 3 up, 3 in 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: pgmap v79: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-10-07T23:10:51.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi123 2023-10-07T23:10:51.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:51.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:51.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:51.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:50 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:10:51.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: osd.2 [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] boot 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: osdmap e18: 3 total, 3 up, 3 in 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: pgmap v79: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi123 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:10:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:10:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:50 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:51.434 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:10:52.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:51 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi123 to 17172M 2023-10-07T23:10:52.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-10-07T23:10:52.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:51 smithi067 ceph-mon[82723]: osdmap e19: 3 total, 3 up, 3 in 2023-10-07T23:10:52.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:10:52.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:51 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi123 to 17172M 2023-10-07T23:10:52.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-10-07T23:10:52.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:51 smithi123 ceph-mon[122926]: osdmap e19: 3 total, 3 up, 3 in 2023-10-07T23:10:52.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:10:52.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:51 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi123 to 17172M 2023-10-07T23:10:52.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-10-07T23:10:52.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:51 smithi086 ceph-mon[122131]: osdmap e19: 3 total, 3 up, 3 in 2023-10-07T23:10:52.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:10:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 -- 172.21.15.67:0/3582476037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fe980 msgr2=0x7f8c3c1052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 --2- 172.21.15.67:0/3582476037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c1052b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009d60 tx=0x7f8c34004d10 comp rx=0 tx=0).stop 2023-10-07T23:10:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 -- 172.21.15.67:0/3582476037 shutdown_connections 2023-10-07T23:10:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 --2- 172.21.15.67:0/3582476037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c1052b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 --2- 172.21.15.67:0/3582476037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fce60 0x7f8c3c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 --2- 172.21.15.67:0/3582476037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 0x7f8c3c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:52.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 -- 172.21.15.67:0/3582476037 >> 172.21.15.67:0/3582476037 conn(0x7f8c3c0f85c0 msgr2=0x7f8c3c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:10:52.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 -- 172.21.15.67:0/3582476037 shutdown_connections 2023-10-07T23:10:52.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.931+0000 7f8c4316c700 1 -- 172.21.15.67:0/3582476037 wait complete. 2023-10-07T23:10:52.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.932+0000 7f8c4316c700 1 Processor -- start 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.932+0000 7f8c4316c700 1 -- start start 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.932+0000 7f8c4316c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fce60 0x7f8c3c10faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.932+0000 7f8c4316c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 0x7f8c3c10ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c4316c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c117070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c4316c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c117680 con 0x7f8c3c0fce60 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c4316c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1177f0 con 0x7f8c3c0fdfd0 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c4316c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1179b0 con 0x7f8c3c0fe980 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c3bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 0x7f8c3c10ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:52.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c3bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 0x7f8c3c10ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:46042/0 (socket says 172.21.15.67:46042) 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c3bfff700 1 -- 172.21.15.67:0/1285087427 learned_addr learned my addr 172.21.15.67:0/1285087427 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c41709700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c117070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c40f08700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fce60 0x7f8c3c10faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c41709700 1 -- 172.21.15.67:0/1285087427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 msgr2=0x7f8c3c10ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c41709700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 0x7f8c3c10ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c41709700 1 -- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fce60 msgr2=0x7f8c3c10faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c41709700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fce60 0x7f8c3c10faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:10:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.933+0000 7f8c41709700 1 -- 172.21.15.67:0/1285087427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34009940 con 0x7f8c3c0fe980 2023-10-07T23:10:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.934+0000 7f8c41709700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c117070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c34015040 tx=0x7f8c34004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:10:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.934+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400e030 con 0x7f8c3c0fe980 2023-10-07T23:10:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.934+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c117c30 con 0x7f8c3c0fe980 2023-10-07T23:10:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.934+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3400bec0 con 0x7f8c3c0fe980 2023-10-07T23:10:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.934+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34012680 con 0x7f8c3c0fe980 2023-10-07T23:10:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.934+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c118090 con 0x7f8c3c0fe980 2023-10-07T23:10:52.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.935+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c108670 con 0x7f8c3c0fe980 2023-10-07T23:10:52.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.937+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) v1 ==== 129283+0+0 (secure 0 0 0) 0x7f8c34012ab0 con 0x7f8c3c0fe980 2023-10-07T23:10:52.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.938+0000 7f8c397fa700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8c2c09b6e0 0x7f8c2c09db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:10:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.938+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3233+0+0 (secure 0 0 0) 0x7f8c340c5c60 con 0x7f8c3c0fe980 2023-10-07T23:10:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.938+0000 7f8c40f08700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8c2c09b6e0 0x7f8c2c09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:10:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.939+0000 7f8c40f08700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8c2c09b6e0 0x7f8c2c09db90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009b80 tx=0x7f8c3000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:10:52.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:52.942+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8c34092880 con 0x7f8c3c0fe980 2023-10-07T23:10:53.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-07T23:10:53.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:52 smithi067 ceph-mon[82723]: osdmap e20: 3 total, 3 up, 3 in 2023-10-07T23:10:53.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:52 smithi067 ceph-mon[82723]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:53.005 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:52 smithi067 sudo[108105]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-07T23:10:53.005 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:52 smithi067 sudo[108105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:53.005 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:52 smithi067 sudo[108105]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:53.006 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:52 smithi067 sudo[108105]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:53.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-07T23:10:53.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:52 smithi123 ceph-mon[122926]: osdmap e20: 3 total, 3 up, 3 in 2023-10-07T23:10:53.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:52 smithi123 ceph-mon[122926]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:53.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:53.153+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c3c070e80 con 0x7f8c2c09b6e0 2023-10-07T23:10:53.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-07T23:10:53.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:52 smithi086 ceph-mon[122131]: osdmap e20: 3 total, 3 up, 3 in 2023-10-07T23:10:53.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:52 smithi086 ceph-mon[122131]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:53.504 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:53 smithi067 sudo[108164]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-07T23:10:53.504 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:53 smithi067 sudo[108164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:53.504 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:53 smithi067 sudo[108164]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:53.504 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:10:53 smithi067 sudo[108164]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:53.728 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134397]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-07T23:10:53.728 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:53.728 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134397]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:53.728 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134397]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:54.140 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:53 smithi123 sudo[135874]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-07T23:10:54.140 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:53 smithi123 sudo[135874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:54.141 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:53 smithi123 sudo[135874]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:54.141 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:54 smithi123 sudo[135874]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: osdmap e21: 3 total, 3 up, 3 in 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:54.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:53 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:54.197 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134400]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-07T23:10:54.197 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134400]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:54.197 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134400]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:54.198 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:10:53 smithi086 sudo[134400]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: osdmap e21: 3 total, 3 up, 3 in 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:53 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:54.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: osdmap e21: 3 total, 3 up, 3 in 2023-10-07T23:10:54.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:10:54.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:10:54.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:10:54.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:10:54.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:54.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:53 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:54.640 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:54 smithi123 sudo[135877]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-07T23:10:54.641 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:54 smithi123 sudo[135877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:54.641 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:54 smithi123 sudo[135877]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:54.641 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:10:54 smithi123 sudo[135877]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:54.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:54 smithi067 sudo[108433]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-07T23:10:54.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:54 smithi067 sudo[108433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:54.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:54 smithi067 sudo[108433]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:54.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:54 smithi067 sudo[108433]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:55.387 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:55 smithi067 sudo[108509]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-07T23:10:55.387 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:55 smithi067 sudo[108509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:55.387 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:55 smithi067 sudo[108509]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:55.387 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:55 smithi067 sudo[108509]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:55.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:55 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:55.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:55 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:55.876 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135880]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-07T23:10:55.876 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135880]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:55.876 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135880]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:55.877 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135880]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:55.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:10:55.884+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f8c34092c50 con 0x7f8c3c0fe980 2023-10-07T23:10:56.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135883]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-07T23:10:56.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135883]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:56.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135883]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:56.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 sudo[135883]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:56.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:55 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:10:56.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134404]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-07T23:10:56.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:56.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134404]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:56.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134404]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:56.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134407]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-07T23:10:56.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134407]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 sudo[134407]: pam_unix(sudo:session): session closed for user root 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3100484058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b364f2c-b808-4c14-b266-ae3dbe82c90a"}]: dispatch 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3100484058' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b364f2c-b808-4c14-b266-ae3dbe82c90a"}]': finished 2023-10-07T23:10:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: osdmap e22: 4 total, 3 up, 4 in 2023-10-07T23:10:56.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:10:56.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:56.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3100484058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b364f2c-b808-4c14-b266-ae3dbe82c90a"}]: dispatch 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3100484058' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b364f2c-b808-4c14-b266-ae3dbe82c90a"}]': finished 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: osdmap e22: 4 total, 3 up, 4 in 2023-10-07T23:10:56.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:10:56.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3100484058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b364f2c-b808-4c14-b266-ae3dbe82c90a"}]: dispatch 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3100484058' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b364f2c-b808-4c14-b266-ae3dbe82c90a"}]': finished 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: osdmap e22: 4 total, 3 up, 4 in 2023-10-07T23:10:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:10:57.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:57 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/78968266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:57.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:57 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/78968266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:57.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:57 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/78968266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:10:58.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:10:58 smithi086 ceph-mon[122131]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:58.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:10:58 smithi067 ceph-mon[82723]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:10:58.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:10:58 smithi123 ceph-mon[122926]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:01.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:00 smithi086 ceph-mon[122131]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:01.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:00 smithi067 ceph-mon[82723]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:01.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:00 smithi123 ceph-mon[122926]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:03.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:02 smithi086 ceph-mon[122131]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 48 KiB/s wr, 4 op/s 2023-10-07T23:11:03.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:02 smithi067 ceph-mon[82723]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 48 KiB/s wr, 4 op/s 2023-10-07T23:11:03.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:02 smithi123 ceph-mon[122926]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 48 KiB/s wr, 4 op/s 2023-10-07T23:11:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-07T23:11:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-07T23:11:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:04.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-07T23:11:04.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:04 smithi086 ceph-mon[122131]: Deploying daemon osd.3 on smithi067 2023-10-07T23:11:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:04 smithi086 ceph-mon[122131]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 1.2 KiB/s rd, 35 KiB/s wr, 3 op/s 2023-10-07T23:11:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:04 smithi067 ceph-mon[82723]: Deploying daemon osd.3 on smithi067 2023-10-07T23:11:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:04 smithi067 ceph-mon[82723]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 1.2 KiB/s rd, 35 KiB/s wr, 3 op/s 2023-10-07T23:11:05.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:04 smithi123 ceph-mon[122926]: Deploying daemon osd.3 on smithi067 2023-10-07T23:11:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:04 smithi123 ceph-mon[122926]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 1.2 KiB/s rd, 35 KiB/s wr, 3 op/s 2023-10-07T23:11:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:06 smithi086 ceph-mon[122131]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 970 B/s rd, 28 KiB/s wr, 2 op/s 2023-10-07T23:11:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:06 smithi067 ceph-mon[82723]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 970 B/s rd, 28 KiB/s wr, 2 op/s 2023-10-07T23:11:07.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:06 smithi123 ceph-mon[122926]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 970 B/s rd, 28 KiB/s wr, 2 op/s 2023-10-07T23:11:09.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:08 smithi067 ceph-mon[82723]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 801 B/s rd, 23 KiB/s wr, 2 op/s 2023-10-07T23:11:09.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:08 smithi086 ceph-mon[122131]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 801 B/s rd, 23 KiB/s wr, 2 op/s 2023-10-07T23:11:09.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:08 smithi123 ceph-mon[122926]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 801 B/s rd, 23 KiB/s wr, 2 op/s 2023-10-07T23:11:11.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:11.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:11.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:10 smithi067 ceph-mon[82723]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-07T23:11:11.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:11.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:11.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:10 smithi123 ceph-mon[122926]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-07T23:11:11.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:11.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:11.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:11.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:10 smithi086 ceph-mon[122131]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-07T23:11:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.072+0000 7f8c397fa700 1 -- 172.21.15.67:0/1285087427 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c3c070e80 con 0x7f8c2c09b6e0 2023-10-07T23:11:13.074 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8c2c09b6e0 msgr2=0x7f8c2c09db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8c2c09b6e0 0x7f8c2c09db90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009b80 tx=0x7f8c3000b040 comp rx=0 tx=0).stop 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fe980 msgr2=0x7f8c3c117070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c117070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c34015040 tx=0x7f8c34004d10 comp rx=0 tx=0).stop 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 shutdown_connections 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8c2c09b6e0 0x7f8c2c09db90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c0fe980 0x7f8c3c117070 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c3c0fce60 0x7f8c3c10faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 --2- 172.21.15.67:0/1285087427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c3c0fdfd0 0x7f8c3c10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 >> 172.21.15.67:0/1285087427 conn(0x7f8c3c0f85c0 msgr2=0x7f8c3c0fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:11:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 shutdown_connections 2023-10-07T23:11:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:11:13.075+0000 7f8c4316c700 1 -- 172.21.15.67:0/1285087427 wait complete. 2023-10-07T23:11:13.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:12 smithi086 ceph-mon[122131]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-07T23:11:13.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.228 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:12 smithi067 ceph-mon[82723]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-07T23:11:13.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:13.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:13.229 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:12 smithi123 ceph-mon[122926]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-07T23:11:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:13.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:13.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:13.519 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-07T23:11:13.524 INFO:tasks.cephadm:Deploying osd.4 on smithi086 with /dev/vg_nvme/lv_3... 2023-10-07T23:11:13.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-07T23:11:13.593 INFO:journalctl@ceph.osd.3.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:11:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:11:14.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:14.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:14.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:14.176 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:13 smithi067 ceph-mon[82723]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-07T23:11:14.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:14.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:14.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:14.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:13 smithi086 ceph-mon[122131]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-07T23:11:14.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:14.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:14.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:14.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:13 smithi123 ceph-mon[122926]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-07T23:11:15.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:15 smithi123 ceph-mon[122926]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:15.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:15 smithi123 ceph-mon[122926]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-07T23:11:15.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:15 smithi123 ceph-mon[122926]: osdmap e23: 4 total, 3 up, 4 in 2023-10-07T23:11:15.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:15.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:15 smithi123 ceph-mon[122926]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:11:15.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:15 smithi086 ceph-mon[122131]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:15.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:15 smithi086 ceph-mon[122131]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-07T23:11:15.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:15 smithi086 ceph-mon[122131]: osdmap e23: 4 total, 3 up, 4 in 2023-10-07T23:11:15.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:15.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:15 smithi086 ceph-mon[122131]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:11:15.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:15 smithi067 ceph-mon[82723]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:15.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:15 smithi067 ceph-mon[82723]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-07T23:11:15.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:15 smithi067 ceph-mon[82723]: osdmap e23: 4 total, 3 up, 4 in 2023-10-07T23:11:15.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:15.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:15 smithi067 ceph-mon[82723]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:11:15.505 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:11:15 smithi067 conmon[111399]: 2023-10-07T23:11:15.052+0000 7f4a0e3c6700 -1 osd.3 0 waiting for initial osdmap 2023-10-07T23:11:15.505 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:11:15 smithi067 conmon[111399]: 2023-10-07T23:11:15.059+0000 7f4a0a1c3700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:11:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-07T23:11:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-07T23:11:16.006 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2023-10-07T23:11:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2023-10-07T23:11:16.006 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011123 s, 943 MB/s 2023-10-07T23:11:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2023-10-07T23:11:16.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:16 smithi123 ceph-mon[122926]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:11:16.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:16 smithi123 ceph-mon[122926]: osdmap e24: 4 total, 3 up, 4 in 2023-10-07T23:11:16.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:16.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:16 smithi086 ceph-mon[122131]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:11:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:16 smithi086 ceph-mon[122131]: osdmap e24: 4 total, 3 up, 4 in 2023-10-07T23:11:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:16.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:16 smithi067 ceph-mon[82723]: from='osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:11:16.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:16 smithi067 ceph-mon[82723]: osdmap e24: 4 total, 3 up, 4 in 2023-10-07T23:11:16.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:16.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:16.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2023-10-07T23:11:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] boot 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: osdmap e25: 4 total, 4 up, 4 in 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:17.383 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:17.384 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:17 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:11:17.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:11:17.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] boot 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: osdmap e25: 4 total, 4 up, 4 in 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:17.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:17 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:11:17.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:11:17.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2023-10-07T23:11:17.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: osd.3 [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] boot 2023-10-07T23:11:17.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: osdmap e25: 4 total, 4 up, 4 in 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:17.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:17.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:17.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:17 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:18.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:18 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi067 2023-10-07T23:11:18.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:18 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi067 to 8633M 2023-10-07T23:11:18.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:18 smithi086 ceph-mon[122131]: osdmap e26: 4 total, 4 up, 4 in 2023-10-07T23:11:18.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:18 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi067 2023-10-07T23:11:18.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:18 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi067 to 8633M 2023-10-07T23:11:18.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:18 smithi123 ceph-mon[122926]: osdmap e26: 4 total, 4 up, 4 in 2023-10-07T23:11:18.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:18 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi067 2023-10-07T23:11:18.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:18 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi067 to 8633M 2023-10-07T23:11:18.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:18 smithi067 ceph-mon[82723]: osdmap e26: 4 total, 4 up, 4 in 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 -- 172.21.15.86:0/2310929700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0100030 msgr2=0x7febb0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 --2- 172.21.15.86:0/2310929700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0100030 0x7febb0102430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feba00049c0 tx=0x7feba0009e30 comp rx=0 tx=0).stop 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 -- 172.21.15.86:0/2310929700 shutdown_connections 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 --2- 172.21.15.86:0/2310929700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febb0102ac0 0x7febb0104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 --2- 172.21.15.86:0/2310929700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febb00662f0 0x7febb00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 --2- 172.21.15.86:0/2310929700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0100030 0x7febb0102430 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 -- 172.21.15.86:0/2310929700 >> 172.21.15.86:0/2310929700 conn(0x7febb00f74d0 msgr2=0x7febb00f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:11:18.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 -- 172.21.15.86:0/2310929700 shutdown_connections 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 -- 172.21.15.86:0/2310929700 wait complete. 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.726+0000 7febb869d700 1 Processor -- start 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 -- start start 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febb00662f0 0x7febb010db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febb0100030 0x7febb010e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 0x7febb0115120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb0115730 con 0x7febb0100030 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb01158a0 con 0x7febb0102ac0 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb0115a60 con 0x7febb00662f0 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 0x7febb0115120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:18.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 0x7febb0115120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47768/0 (socket says 172.21.15.86:47768) 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 -- 172.21.15.86:0/3207758393 learned_addr learned my addr 172.21.15.86:0/3207758393 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6439700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febb00662f0 0x7febb010db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 -- 172.21.15.86:0/3207758393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febb00662f0 msgr2=0x7febb010db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febb00662f0 0x7febb010db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 -- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febb0100030 msgr2=0x7febb010e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febb0100030 0x7febb010e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 -- 172.21.15.86:0/3207758393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0005040 con 0x7febb0102ac0 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb6c3a700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 0x7febb0115120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feba800bc30 tx=0x7feba8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80092e0 con 0x7febb0102ac0 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feba800cc70 con 0x7febb0102ac0 2023-10-07T23:11:18.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb0115d40 con 0x7febb0102ac0 2023-10-07T23:11:18.731 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8015d50 con 0x7febb0102ac0 2023-10-07T23:11:18.731 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.727+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb01161d0 con 0x7febb0102ac0 2023-10-07T23:11:18.731 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.728+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb00f9000 con 0x7febb0102ac0 2023-10-07T23:11:18.732 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.729+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7feba8012070 con 0x7febb0102ac0 2023-10-07T23:11:18.732 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.730+0000 7feba6ffd700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feb9c09b7f0 0x7feb9c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:18.732 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.730+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3694+0+0 (secure 0 0 0) 0x7feba80c3a20 con 0x7febb0102ac0 2023-10-07T23:11:18.732 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.730+0000 7febb6439700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feb9c09b7f0 0x7feb9c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:18.732 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.731+0000 7febb6439700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feb9c09b7f0 0x7feb9c09dca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febac004c40 tx=0x7febac009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:11:18.733 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.734+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7feba8090440 con 0x7febb0102ac0 2023-10-07T23:11:18.926 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:18.926+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7febb002b4d0 con 0x7feb9c09b7f0 2023-10-07T23:11:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:19 smithi123 ceph-mon[122926]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:19 smithi123 ceph-mon[122926]: osdmap e27: 4 total, 4 up, 4 in 2023-10-07T23:11:19.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:19 smithi123 ceph-mon[122926]: osdmap e28: 4 total, 4 up, 4 in 2023-10-07T23:11:19.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:19 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:11:19.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:19 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:11:19.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:19 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:19 smithi086 ceph-mon[122131]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:19 smithi086 ceph-mon[122131]: osdmap e27: 4 total, 4 up, 4 in 2023-10-07T23:11:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:19 smithi086 ceph-mon[122131]: osdmap e28: 4 total, 4 up, 4 in 2023-10-07T23:11:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:19 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:11:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:19 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:11:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:19 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:19.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:19 smithi067 ceph-mon[82723]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:19.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:19 smithi067 ceph-mon[82723]: osdmap e27: 4 total, 4 up, 4 in 2023-10-07T23:11:19.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:19 smithi067 ceph-mon[82723]: osdmap e28: 4 total, 4 up, 4 in 2023-10-07T23:11:19.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:19 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:11:19.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:19 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:11:19.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:19 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:20.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:20 smithi123 ceph-mon[122926]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:11:20.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:20 smithi123 ceph-mon[122926]: osdmap e29: 4 total, 4 up, 4 in 2023-10-07T23:11:20.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:20 smithi086 ceph-mon[122131]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:11:20.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:20 smithi086 ceph-mon[122131]: osdmap e29: 4 total, 4 up, 4 in 2023-10-07T23:11:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:20 smithi067 ceph-mon[82723]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:11:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:20 smithi067 ceph-mon[82723]: osdmap e29: 4 total, 4 up, 4 in 2023-10-07T23:11:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:21 smithi123 ceph-mon[122926]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:21.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:21 smithi086 ceph-mon[122131]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:21.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:21 smithi067 ceph-mon[82723]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:22 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/730801273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]: dispatch 2023-10-07T23:11:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:22 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]: dispatch 2023-10-07T23:11:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:22 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]': finished 2023-10-07T23:11:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:22 smithi123 ceph-mon[122926]: osdmap e30: 5 total, 4 up, 5 in 2023-10-07T23:11:22.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:22.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:22 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/730801273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]: dispatch 2023-10-07T23:11:22.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:22 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]: dispatch 2023-10-07T23:11:22.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:22 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]': finished 2023-10-07T23:11:22.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:22 smithi086 ceph-mon[122131]: osdmap e30: 5 total, 4 up, 5 in 2023-10-07T23:11:22.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:22.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:22 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/730801273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]: dispatch 2023-10-07T23:11:22.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:22 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]: dispatch 2023-10-07T23:11:22.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:22 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75a0c533-a0cd-426d-9820-189b07ecef7b"}]': finished 2023-10-07T23:11:22.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:22 smithi067 ceph-mon[82723]: osdmap e30: 5 total, 4 up, 5 in 2023-10-07T23:11:22.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:23 smithi123 ceph-mon[122926]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-07T23:11:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:23 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/3441730717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:11:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:23 smithi086 ceph-mon[122131]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-07T23:11:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:23 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/3441730717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:11:23.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:23 smithi067 ceph-mon[82723]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-07T23:11:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:23 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/3441730717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:11:25.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:24 smithi123 ceph-mon[122926]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-10-07T23:11:25.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:24 smithi086 ceph-mon[122131]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-10-07T23:11:25.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:24 smithi067 ceph-mon[82723]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-10-07T23:11:27.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:26 smithi086 ceph-mon[122131]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-10-07T23:11:27.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:26 smithi067 ceph-mon[82723]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-10-07T23:11:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:26 smithi123 ceph-mon[122926]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-10-07T23:11:29.195 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:28 smithi086 ceph-mon[122131]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-10-07T23:11:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:28 smithi067 ceph-mon[82723]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-10-07T23:11:29.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:28 smithi123 ceph-mon[122926]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-10-07T23:11:30.102 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-07T23:11:30.102 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:30.102 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:29 smithi086 ceph-mon[122131]: Deploying daemon osd.4 on smithi086 2023-10-07T23:11:30.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-07T23:11:30.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:30.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:29 smithi067 ceph-mon[82723]: Deploying daemon osd.4 on smithi086 2023-10-07T23:11:30.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-07T23:11:30.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:30.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:29 smithi123 ceph-mon[122926]: Deploying daemon osd.4 on smithi086 2023-10-07T23:11:31.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:30 smithi086 ceph-mon[122131]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-07T23:11:31.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:30 smithi123 ceph-mon[122926]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-07T23:11:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:30 smithi067 ceph-mon[82723]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-07T23:11:33.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:32 smithi086 ceph-mon[122131]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:33.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:32 smithi067 ceph-mon[82723]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:33.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:32 smithi123 ceph-mon[122926]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:35.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:34 smithi123 ceph-mon[122926]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:35.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:35.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:35.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:35.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:34 smithi086 ceph-mon[122131]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:35.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:35.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:35.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:35.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:34 smithi067 ceph-mon[82723]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:35.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:35.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:35.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:36.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:36 smithi086 ceph-mon[122131]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:37.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:36 smithi067 ceph-mon[82723]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:37.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:36 smithi123 ceph-mon[122926]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:38.050 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 4 on host 'smithi086' 2023-10-07T23:11:38.050 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.049+0000 7feba6ffd700 1 -- 172.21.15.86:0/3207758393 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7febb002b4d0 con 0x7feb9c09b7f0 2023-10-07T23:11:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feb9c09b7f0 msgr2=0x7feb9c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feb9c09b7f0 0x7feb9c09dca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febac004c40 tx=0x7febac009040 comp rx=0 tx=0).stop 2023-10-07T23:11:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 msgr2=0x7febb0115120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 0x7febb0115120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feba800bc30 tx=0x7feba8009840 comp rx=0 tx=0).stop 2023-10-07T23:11:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 shutdown_connections 2023-10-07T23:11:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febb0102ac0 0x7febb0115120 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feb9c09b7f0 0x7feb9c09dca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febb00662f0 0x7febb010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 --2- 172.21.15.86:0/3207758393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febb0100030 0x7febb010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.051+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 >> 172.21.15.86:0/3207758393 conn(0x7febb00f74d0 msgr2=0x7febb00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:11:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.052+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 shutdown_connections 2023-10-07T23:11:38.054 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:11:38.052+0000 7febb869d700 1 -- 172.21.15.86:0/3207758393 wait complete. 2023-10-07T23:11:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:38.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:38.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:38.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:38.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:38.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:11:38.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:38.770 DEBUG:teuthology.orchestra.run.smithi086:osd.4> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-07T23:11:38.772 INFO:tasks.cephadm:Deploying osd.5 on smithi123 with /dev/vg_nvme/lv_3... 2023-10-07T23:11:38.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-07T23:11:38.814 INFO:journalctl@ceph.osd.4.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:11:39.373 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:11:39.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:39 smithi086 ceph-mon[122131]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:39 smithi086 ceph-mon[122131]: from='osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-07T23:11:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:39 smithi086 ceph-mon[122131]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-07T23:11:39.587 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:39 smithi123 ceph-mon[122926]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:39.587 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:39 smithi123 ceph-mon[122926]: from='osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-07T23:11:39.587 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:39 smithi123 ceph-mon[122926]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-07T23:11:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:39 smithi067 ceph-mon[82723]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:39 smithi067 ceph-mon[82723]: from='osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-07T23:11:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:39 smithi067 ceph-mon[82723]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-07T23:11:40.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:40 smithi123 ceph-mon[122926]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-07T23:11:40.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:40 smithi123 ceph-mon[122926]: osdmap e31: 5 total, 4 up, 5 in 2023-10-07T23:11:40.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:40 smithi123 ceph-mon[122926]: from='osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:11:40.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:40 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:40.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:40 smithi123 ceph-mon[122926]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:11:40.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:40 smithi086 ceph-mon[122131]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-07T23:11:40.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:40 smithi086 ceph-mon[122131]: osdmap e31: 5 total, 4 up, 5 in 2023-10-07T23:11:40.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:40 smithi086 ceph-mon[122131]: from='osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:11:40.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:40 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:40.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:40 smithi086 ceph-mon[122131]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:11:40.698 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:11:40 smithi086 conmon[138642]: 2023-10-07T23:11:40.318+0000 7fd70f6d5700 -1 osd.4 0 waiting for initial osdmap 2023-10-07T23:11:40.698 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:11:40 smithi086 conmon[138642]: 2023-10-07T23:11:40.325+0000 7fd7084cc700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:11:40.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:40 smithi067 ceph-mon[82723]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-07T23:11:40.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:40 smithi067 ceph-mon[82723]: osdmap e31: 5 total, 4 up, 5 in 2023-10-07T23:11:40.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:40 smithi067 ceph-mon[82723]: from='osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:11:40.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:40 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:40.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:40 smithi067 ceph-mon[82723]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:11:41.268 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-07T23:11:41.268 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-07T23:11:41.268 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-10-07T23:11:41.268 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-10-07T23:11:41.268 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109443 s, 958 MB/s 2023-10-07T23:11:41.269 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-10-07T23:11:41.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:41 smithi123 ceph-mon[122926]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:41.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:41 smithi123 ceph-mon[122926]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:11:41.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:41 smithi123 ceph-mon[122926]: osdmap e32: 5 total, 4 up, 5 in 2023-10-07T23:11:41.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:41 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:41 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:41 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:41 smithi086 ceph-mon[122131]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:41 smithi086 ceph-mon[122131]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:11:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:41 smithi086 ceph-mon[122131]: osdmap e32: 5 total, 4 up, 5 in 2023-10-07T23:11:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:41 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:41 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:41 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:41 smithi067 ceph-mon[82723]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:41.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:41 smithi067 ceph-mon[82723]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:11:41.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:41 smithi067 ceph-mon[82723]: osdmap e32: 5 total, 4 up, 5 in 2023-10-07T23:11:41.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:41 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:41 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:41 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:41.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2023-10-07T23:11:42.482 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:11:42.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] boot 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: osdmap e33: 5 total, 5 up, 5 in 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:42.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:42.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:11:42.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] boot 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: osdmap e33: 5 total, 5 up, 5 in 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:42.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: osd.4 [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] boot 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: osdmap e33: 5 total, 5 up, 5 in 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:42.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:11:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:11:42.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:11:43.591 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:43 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi086 2023-10-07T23:11:43.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:43 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi086 to 8586M 2023-10-07T23:11:43.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:43 smithi123 ceph-mon[122926]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:43.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:43 smithi123 ceph-mon[122926]: osdmap e34: 5 total, 5 up, 5 in 2023-10-07T23:11:43.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:43 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi086 2023-10-07T23:11:43.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:43 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi086 to 8586M 2023-10-07T23:11:43.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:43 smithi086 ceph-mon[122131]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:43.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:43 smithi086 ceph-mon[122131]: osdmap e34: 5 total, 5 up, 5 in 2023-10-07T23:11:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:43 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi086 2023-10-07T23:11:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:43 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi086 to 8586M 2023-10-07T23:11:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:43 smithi067 ceph-mon[82723]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2023-10-07T23:11:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:43 smithi067 ceph-mon[82723]: osdmap e34: 5 total, 5 up, 5 in 2023-10-07T23:11:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 -- 172.21.15.123:0/1899590337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 msgr2=0x7f74301006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- 172.21.15.123:0/1899590337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301006c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f74200049c0 tx=0x7f7420009e30 comp rx=0 tx=0).stop 2023-10-07T23:11:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 -- 172.21.15.123:0/1899590337 shutdown_connections 2023-10-07T23:11:44.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- 172.21.15.123:0/1899590337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7430100c00 0x7f74301074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- 172.21.15.123:0/1899590337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74300ff060 0x7f74300ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- 172.21.15.123:0/1899590337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301006c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 -- 172.21.15.123:0/1899590337 >> 172.21.15.123:0/1899590337 conn(0x7f74300fa7c0 msgr2=0x7f74300fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 -- 172.21.15.123:0/1899590337 shutdown_connections 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 -- 172.21.15.123:0/1899590337 wait complete. 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 Processor -- start 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 -- start start 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74300ff060 0x7f743010fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.089+0000 7f74386f7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7430100c00 0x7f7430117250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:44.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f74386f7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74301178a0 con 0x7f7430100c00 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f74386f7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430117a10 con 0x7f74300ff060 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f74386f7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430117bd0 con 0x7f7430100270 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37738/0 (socket says 172.21.15.123:37738) 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 -- 172.21.15.123:0/774470181 learned_addr learned my addr 172.21.15.123:0/774470181 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7436493700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74300ff060 0x7f743010fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7436c94700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7430100c00 0x7f7430117250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 -- 172.21.15.123:0/774470181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74300ff060 msgr2=0x7f743010fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74300ff060 0x7f743010fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 -- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7430100c00 msgr2=0x7f7430117250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:11:44.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7430100c00 0x7f7430117250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 -- 172.21.15.123:0/774470181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420005040 con 0x7f7430100270 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7436493700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74300ff060 0x7f743010fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7436c94700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7430100c00 0x7f7430117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7435c92700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301101d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7420007f30 tx=0x7f7420009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000e040 con 0x7f7430100270 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7420013420 con 0x7f7430100270 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7430117e50 con 0x7f7430100270 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742001c660 con 0x7f7430100270 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.090+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74301182d0 con 0x7f7430100270 2023-10-07T23:11:44.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.091+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7430106560 con 0x7f7430100270 2023-10-07T23:11:44.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.092+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f7420013590 con 0x7f7430100270 2023-10-07T23:11:44.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.092+0000 7f7426ffd700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f741c09b770 0x7f741c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:11:44.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.093+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4097+0+0 (secure 0 0 0) 0x7f74200c5230 con 0x7f7430100270 2023-10-07T23:11:44.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.093+0000 7f7436493700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f741c09b770 0x7f741c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:11:44.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.093+0000 7f7436493700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f741c09b770 0x7f741c09dc20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7430110b80 tx=0x7f742c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:11:44.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.097+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7420091ab0 con 0x7f7430100270 2023-10-07T23:11:44.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:11:44.286+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f743002b4c0 con 0x7f741c09b770 2023-10-07T23:11:45.059 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:44 smithi123 ceph-mon[122926]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:45.059 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:44 smithi123 ceph-mon[122926]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:11:45.060 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:44 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:11:45.060 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:44 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:11:45.060 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:44 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:45.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:44 smithi086 ceph-mon[122131]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:44 smithi086 ceph-mon[122131]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:11:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:44 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:11:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:44 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:11:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:44 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:44 smithi067 ceph-mon[82723]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:44 smithi067 ceph-mon[82723]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:11:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:44 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:11:45.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:44 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:11:45.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:44 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:46 smithi086 ceph-mon[122131]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:46 smithi067 ceph-mon[82723]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:47.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:46 smithi123 ceph-mon[122926]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:48.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:47 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/2715113089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]: dispatch 2023-10-07T23:11:48.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:47 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]: dispatch 2023-10-07T23:11:48.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:47 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]': finished 2023-10-07T23:11:48.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:47 smithi086 ceph-mon[122131]: osdmap e35: 6 total, 5 up, 6 in 2023-10-07T23:11:48.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:11:48.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:47 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/2321497174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:11:48.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:47 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/2715113089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]: dispatch 2023-10-07T23:11:48.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:47 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]: dispatch 2023-10-07T23:11:48.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:47 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]': finished 2023-10-07T23:11:48.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:47 smithi067 ceph-mon[82723]: osdmap e35: 6 total, 5 up, 6 in 2023-10-07T23:11:48.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:11:48.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:47 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/2321497174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:11:48.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:47 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/2715113089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]: dispatch 2023-10-07T23:11:48.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:47 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]: dispatch 2023-10-07T23:11:48.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:47 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1266769f-5b0e-44be-adc4-2175cdb40312"}]': finished 2023-10-07T23:11:48.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:47 smithi123 ceph-mon[122926]: osdmap e35: 6 total, 5 up, 6 in 2023-10-07T23:11:48.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:11:48.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:47 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/2321497174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:11:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:49 smithi123 ceph-mon[122926]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:49 smithi086 ceph-mon[122131]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:49 smithi067 ceph-mon[82723]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:51.388 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:51 smithi123 ceph-mon[122926]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:51.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:51 smithi086 ceph-mon[122131]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:51.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:51 smithi067 ceph-mon[82723]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:53.366 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:53 smithi123 ceph-mon[122926]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:53 smithi086 ceph-mon[122131]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:53.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:53 smithi067 ceph-mon[82723]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:55.075 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:54 smithi123 ceph-mon[122926]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:55.075 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-07T23:11:55.075 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:55.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:54 smithi086 ceph-mon[122131]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:55.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-07T23:11:55.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:55.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:54 smithi067 ceph-mon[82723]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:55.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-07T23:11:55.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:11:56.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:55 smithi123 ceph-mon[122926]: Deploying daemon osd.5 on smithi123 2023-10-07T23:11:56.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:55 smithi086 ceph-mon[122131]: Deploying daemon osd.5 on smithi123 2023-10-07T23:11:56.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:55 smithi067 ceph-mon[82723]: Deploying daemon osd.5 on smithi123 2023-10-07T23:11:56.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:56 smithi123 ceph-mon[122926]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:57.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:56 smithi086 ceph-mon[122131]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:57.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:56 smithi067 ceph-mon[82723]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:59.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:11:58 smithi086 ceph-mon[122131]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:11:58 smithi067 ceph-mon[82723]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:11:59.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:11:58 smithi123 ceph-mon[122926]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:00.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:00.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:00.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:00.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:00 smithi123 ceph-mon[122926]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:01.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:01.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:01.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:00 smithi067 ceph-mon[82723]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:01.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:01.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:01.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:00 smithi086 ceph-mon[122131]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:03.214 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 5 on host 'smithi123' 2023-10-07T23:12:03.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.213+0000 7f7426ffd700 1 -- 172.21.15.123:0/774470181 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f743002b4c0 con 0x7f741c09b770 2023-10-07T23:12:03.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f741c09b770 msgr2=0x7f741c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:03.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f741c09b770 0x7f741c09dc20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7430110b80 tx=0x7f742c009040 comp rx=0 tx=0).stop 2023-10-07T23:12:03.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 msgr2=0x7f74301101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:03.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301101d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7420007f30 tx=0x7f7420009e30 comp rx=0 tx=0).stop 2023-10-07T23:12:03.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 shutdown_connections 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74300ff060 0x7f743010fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f741c09b770 0x7f741c09dc20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7430100c00 0x7f7430117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 --2- 172.21.15.123:0/774470181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7430100270 0x7f74301101d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 >> 172.21.15.123:0/774470181 conn(0x7f74300fa7c0 msgr2=0x7f74301036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 shutdown_connections 2023-10-07T23:12:03.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:12:03.216+0000 7f74386f7700 1 -- 172.21.15.123:0/774470181 wait complete. 2023-10-07T23:12:03.433 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:03.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:03.937 DEBUG:teuthology.orchestra.run.smithi123:osd.5> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-07T23:12:03.939 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2023-10-07T23:12:03.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-07T23:12:03.966 INFO:journalctl@ceph.osd.5.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:12:04.464 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:04 smithi123 ceph-mon[122926]: from='osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-07T23:12:04.464 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:04 smithi123 ceph-mon[122926]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-07T23:12:04.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:04 smithi086 ceph-mon[122131]: from='osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-07T23:12:04.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:04 smithi086 ceph-mon[122131]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-07T23:12:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:12:04.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:04 smithi067 ceph-mon[82723]: from='osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-07T23:12:04.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:04 smithi067 ceph-mon[82723]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-07T23:12:05.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:05 smithi123 ceph-mon[122926]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:05.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:05 smithi123 ceph-mon[122926]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-07T23:12:05.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:05 smithi123 ceph-mon[122926]: osdmap e36: 6 total, 5 up, 6 in 2023-10-07T23:12:05.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:05 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:05.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:05 smithi123 ceph-mon[122926]: from='osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:12:05.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:05 smithi123 ceph-mon[122926]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:12:05.373 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:12:05 smithi123 conmon[140237]: 2023-10-07T23:12:05.332+0000 7f3a4d9c5700 -1 osd.5 0 waiting for initial osdmap 2023-10-07T23:12:05.373 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:12:05 smithi123 conmon[140237]: 2023-10-07T23:12:05.339+0000 7f3a4a7c4700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:12:05.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:05 smithi086 ceph-mon[122131]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:05.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:05 smithi086 ceph-mon[122131]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-07T23:12:05.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:05 smithi086 ceph-mon[122131]: osdmap e36: 6 total, 5 up, 6 in 2023-10-07T23:12:05.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:05 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:05.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:05 smithi086 ceph-mon[122131]: from='osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:12:05.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:05 smithi086 ceph-mon[122131]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:12:05.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:05 smithi067 ceph-mon[82723]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:05 smithi067 ceph-mon[82723]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-07T23:12:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:05 smithi067 ceph-mon[82723]: osdmap e36: 6 total, 5 up, 6 in 2023-10-07T23:12:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:05 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:05 smithi067 ceph-mon[82723]: from='osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:12:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:05 smithi067 ceph-mon[82723]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:12:06.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:06 smithi123 ceph-mon[122926]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:12:06.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:06 smithi123 ceph-mon[122926]: osdmap e37: 6 total, 5 up, 6 in 2023-10-07T23:12:06.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:06.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:06.644 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-07T23:12:06.644 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-07T23:12:06.644 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2023-10-07T23:12:06.645 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2023-10-07T23:12:06.645 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.043767 s, 240 MB/s 2023-10-07T23:12:06.645 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2023-10-07T23:12:06.680 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:06 smithi067 ceph-mon[82723]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:12:06.680 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:06 smithi067 ceph-mon[82723]: osdmap e37: 6 total, 5 up, 6 in 2023-10-07T23:12:06.680 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:06.681 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:06.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:06 smithi086 ceph-mon[122131]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:12:06.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:06 smithi086 ceph-mon[122131]: osdmap e37: 6 total, 5 up, 6 in 2023-10-07T23:12:06.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:06.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2023-10-07T23:12:07.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:12:07.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:12:07.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] boot 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: osdmap e38: 6 total, 6 up, 6 in 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:07.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] boot 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: osdmap e38: 6 total, 6 up, 6 in 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:07.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:07.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:07.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:07.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:12:07.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:12:07.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: osd.5 [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] boot 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: osdmap e38: 6 total, 6 up, 6 in 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:07.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:07.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:07.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:07.974 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:12:08.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:08 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi123 2023-10-07T23:12:08.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:08 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi123 to 8586M 2023-10-07T23:12:08.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:08 smithi067 ceph-mon[82723]: osdmap e39: 6 total, 6 up, 6 in 2023-10-07T23:12:08.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:08 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi123 2023-10-07T23:12:08.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:08 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi123 to 8586M 2023-10-07T23:12:08.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:08 smithi123 ceph-mon[122926]: osdmap e39: 6 total, 6 up, 6 in 2023-10-07T23:12:08.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:08 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi123 2023-10-07T23:12:08.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:08 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi123 to 8586M 2023-10-07T23:12:08.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:08 smithi086 ceph-mon[122131]: osdmap e39: 6 total, 6 up, 6 in 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 -- 172.21.15.67:0/849215919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 msgr2=0x7f0bd41052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/849215919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd41052b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d60 tx=0x7f0bcc004d10 comp rx=0 tx=0).stop 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 -- 172.21.15.67:0/849215919 shutdown_connections 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/849215919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd41052b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/849215919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd40fce60 0x7f0bd40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/849215919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd40fdfd0 0x7f0bd40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:09.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.333+0000 7f0bdaf72700 1 -- 172.21.15.67:0/849215919 >> 172.21.15.67:0/849215919 conn(0x7f0bd40f85c0 msgr2=0x7f0bd40fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:12:09.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.334+0000 7f0bdaf72700 1 -- 172.21.15.67:0/849215919 shutdown_connections 2023-10-07T23:12:09.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.334+0000 7f0bdaf72700 1 -- 172.21.15.67:0/849215919 wait complete. 2023-10-07T23:12:09.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.334+0000 7f0bdaf72700 1 Processor -- start 2023-10-07T23:12:09.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.334+0000 7f0bdaf72700 1 -- start start 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bdaf72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd40fce60 0x7f0bd4071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bdaf72700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd40fdfd0 0x7f0bd406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bdaf72700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bdaf72700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4070c40 con 0x7f0bd40fe980 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bdaf72700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4070db0 con 0x7f0bd40fce60 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bdaf72700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41179d0 con 0x7f0bd40fdfd0 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd40704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60618/0 (socket says 172.21.15.67:60618) 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 -- 172.21.15.67:0/1717791473 learned_addr learned my addr 172.21.15.67:0/1717791473 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:12:09.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd8d0e700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd40fce60 0x7f0bd4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd3fff700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd40fdfd0 0x7f0bd406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 -- 172.21.15.67:0/1717791473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd40fdfd0 msgr2=0x7f0bd406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd40fdfd0 0x7f0bd406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 -- 172.21.15.67:0/1717791473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd40fce60 msgr2=0x7f0bd4071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd40fce60 0x7f0bd4071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.335+0000 7f0bd950f700 1 -- 172.21.15.67:0/1717791473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc009940 con 0x7f0bd40fe980 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.336+0000 7f0bd950f700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd40704d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00bd40 tx=0x7f0bcc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.336+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00e030 con 0x7f0bd40fe980 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.336+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd4117c20 con 0x7f0bd40fe980 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.336+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bcc008ec0 con 0x7f0bd40fe980 2023-10-07T23:12:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.336+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc012680 con 0x7f0bd40fe980 2023-10-07T23:12:09.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.336+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd41180d0 con 0x7f0bd40fe980 2023-10-07T23:12:09.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.337+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4103b80 con 0x7f0bd40fe980 2023-10-07T23:12:09.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.339+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f0bcc012a70 con 0x7f0bd40fe980 2023-10-07T23:12:09.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.340+0000 7f0bd17fa700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0bc409b770 0x7f0bc409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:09.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.341+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4529+0+0 (secure 0 0 0) 0x7f0bcc0c6190 con 0x7f0bd40fe980 2023-10-07T23:12:09.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.341+0000 7f0bd8d0e700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0bc409b770 0x7f0bc409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.341+0000 7f0bd8d0e700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0bc409b770 0x7f0bc409dc20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009b80 tx=0x7f0bc800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:12:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.345+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f0bcc0927c0 con 0x7f0bd40fe980 2023-10-07T23:12:09.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:09.556+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bd402b420 con 0x7f0bc409b770 2023-10-07T23:12:09.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:09 smithi123 ceph-mon[122926]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:09.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:09 smithi123 ceph-mon[122926]: osdmap e40: 6 total, 6 up, 6 in 2023-10-07T23:12:09.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:09 smithi123 ceph-mon[122926]: osdmap e41: 6 total, 6 up, 6 in 2023-10-07T23:12:09.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:09 smithi086 ceph-mon[122131]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:09.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:09 smithi086 ceph-mon[122131]: osdmap e40: 6 total, 6 up, 6 in 2023-10-07T23:12:09.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:09 smithi086 ceph-mon[122131]: osdmap e41: 6 total, 6 up, 6 in 2023-10-07T23:12:09.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:09 smithi067 ceph-mon[82723]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:09.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:09 smithi067 ceph-mon[82723]: osdmap e40: 6 total, 6 up, 6 in 2023-10-07T23:12:09.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:09 smithi067 ceph-mon[82723]: osdmap e41: 6 total, 6 up, 6 in 2023-10-07T23:12:10.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:12:10.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:12:10.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:10.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:10 smithi067 ceph-mon[82723]: osdmap e42: 6 total, 6 up, 6 in 2023-10-07T23:12:10.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:12:10.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:12:10.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:10.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:10 smithi123 ceph-mon[122926]: osdmap e42: 6 total, 6 up, 6 in 2023-10-07T23:12:10.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:12:10.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:12:10.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:10.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:10 smithi086 ceph-mon[122131]: osdmap e42: 6 total, 6 up, 6 in 2023-10-07T23:12:11.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:11 smithi123 ceph-mon[122926]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:12:11.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:11 smithi123 ceph-mon[122926]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:11.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:11 smithi086 ceph-mon[122131]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:12:11.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:11 smithi086 ceph-mon[122131]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:11.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:11 smithi067 ceph-mon[82723]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:12:11.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:11 smithi067 ceph-mon[82723]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:13.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:13 smithi123 ceph-mon[122926]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-10-07T23:12:13.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:13 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3058307141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bb57d51-c3a7-473e-9258-8f1dfe6c2761"}]: dispatch 2023-10-07T23:12:13.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:13 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3058307141' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bb57d51-c3a7-473e-9258-8f1dfe6c2761"}]': finished 2023-10-07T23:12:13.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:13 smithi123 ceph-mon[122926]: osdmap e43: 7 total, 6 up, 7 in 2023-10-07T23:12:13.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:13.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:13 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3110784291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:12:13.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:13 smithi086 ceph-mon[122131]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-10-07T23:12:13.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:13 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3058307141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bb57d51-c3a7-473e-9258-8f1dfe6c2761"}]: dispatch 2023-10-07T23:12:13.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:13 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3058307141' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bb57d51-c3a7-473e-9258-8f1dfe6c2761"}]': finished 2023-10-07T23:12:13.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:13 smithi086 ceph-mon[122131]: osdmap e43: 7 total, 6 up, 7 in 2023-10-07T23:12:13.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:13.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:13 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3110784291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:12:13.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:13 smithi067 ceph-mon[82723]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-10-07T23:12:13.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:13 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3058307141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bb57d51-c3a7-473e-9258-8f1dfe6c2761"}]: dispatch 2023-10-07T23:12:13.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:13 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3058307141' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bb57d51-c3a7-473e-9258-8f1dfe6c2761"}]': finished 2023-10-07T23:12:13.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:13 smithi067 ceph-mon[82723]: osdmap e43: 7 total, 6 up, 7 in 2023-10-07T23:12:13.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:13.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:13 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3110784291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:12:15.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:14 smithi123 ceph-mon[122926]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-10-07T23:12:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:14 smithi086 ceph-mon[122131]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-10-07T23:12:15.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:14 smithi067 ceph-mon[82723]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-10-07T23:12:17.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:16 smithi067 ceph-mon[82723]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-10-07T23:12:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:16 smithi086 ceph-mon[122131]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-10-07T23:12:17.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:16 smithi123 ceph-mon[122926]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-10-07T23:12:19.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:18 smithi086 ceph-mon[122131]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-10-07T23:12:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:18 smithi067 ceph-mon[82723]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-10-07T23:12:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:18 smithi123 ceph-mon[122926]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-10-07T23:12:20.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:19 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-07T23:12:20.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:19 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:20.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:19 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-07T23:12:20.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:19 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:20.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:19 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-07T23:12:20.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:19 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:21.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:20 smithi067 ceph-mon[82723]: Deploying daemon osd.6 on smithi067 2023-10-07T23:12:21.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:20 smithi067 ceph-mon[82723]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-07T23:12:21.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:20 smithi086 ceph-mon[122131]: Deploying daemon osd.6 on smithi067 2023-10-07T23:12:21.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:20 smithi086 ceph-mon[122131]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-07T23:12:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:20 smithi123 ceph-mon[122926]: Deploying daemon osd.6 on smithi067 2023-10-07T23:12:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:20 smithi123 ceph-mon[122926]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-07T23:12:23.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:22 smithi067 ceph-mon[82723]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:23.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:22 smithi086 ceph-mon[122131]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:22 smithi123 ceph-mon[122926]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:25.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:24 smithi123 ceph-mon[122926]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:24 smithi086 ceph-mon[122131]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:25.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:24 smithi067 ceph-mon[82723]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:27.366 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:27 smithi067 ceph-mon[82723]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:27.366 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:27.366 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:27.366 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:27 smithi123 ceph-mon[122926]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:27 smithi086 ceph-mon[122131]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:29 smithi067 ceph-mon[82723]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:29.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:29.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.263+0000 7f0bd17fa700 1 -- 172.21.15.67:0/1717791473 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0bd402b420 con 0x7f0bc409b770 2023-10-07T23:12:29.264 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0bc409b770 msgr2=0x7f0bc409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0bc409b770 0x7f0bc409dc20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009b80 tx=0x7f0bc800b040 comp rx=0 tx=0).stop 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 msgr2=0x7f0bd40704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd40704d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00bd40 tx=0x7f0bcc004d10 comp rx=0 tx=0).stop 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 shutdown_connections 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd40fce60 0x7f0bd4071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0bc409b770 0x7f0bc409dc20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd40fe980 0x7f0bd40704d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 --2- 172.21.15.67:0/1717791473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd40fdfd0 0x7f0bd406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 >> 172.21.15.67:0/1717791473 conn(0x7f0bd40f85c0 msgr2=0x7f0bd40fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:12:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 shutdown_connections 2023-10-07T23:12:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:12:29.266+0000 7f0bdaf72700 1 -- 172.21.15.67:0/1717791473 wait complete. 2023-10-07T23:12:29.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:29 smithi123 ceph-mon[122926]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:29 smithi086 ceph-mon[122131]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:29.769 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-07T23:12:29.774 INFO:tasks.cephadm:Deploying osd.7 on smithi086 with /dev/vg_nvme/lv_2... 2023-10-07T23:12:29.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-07T23:12:29.841 INFO:journalctl@ceph.osd.6.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:12:30.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:30 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:30.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:30 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:30.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:30 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:30.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:30 smithi067 ceph-mon[82723]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-07T23:12:30.345 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:12:30 smithi067 conmon[120945]: 2023-10-07T23:12:30.072+0000 7fe185992540 -1 osd.6 0 log_to_monitors true 2023-10-07T23:12:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:12:30.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:30 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:30.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:30 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:30.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:30 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:30.515 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:30 smithi086 ceph-mon[122131]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-07T23:12:30.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:30 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:30.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:30 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:30.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:30 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:30.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:30 smithi123 ceph-mon[122926]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-07T23:12:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:31 smithi067 ceph-mon[82723]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:31 smithi067 ceph-mon[82723]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-07T23:12:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:31 smithi067 ceph-mon[82723]: osdmap e44: 7 total, 6 up, 7 in 2023-10-07T23:12:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:31.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:31 smithi067 ceph-mon[82723]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:12:31.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:12:31 smithi067 conmon[120945]: 2023-10-07T23:12:31.254+0000 7fe17c177700 -1 osd.6 0 waiting for initial osdmap 2023-10-07T23:12:31.506 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:12:31 smithi067 conmon[120945]: 2023-10-07T23:12:31.269+0000 7fe178f76700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:12:31.632 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:31 smithi086 ceph-mon[122131]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:31.632 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:31 smithi086 ceph-mon[122131]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-07T23:12:31.632 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:31 smithi086 ceph-mon[122131]: osdmap e44: 7 total, 6 up, 7 in 2023-10-07T23:12:31.632 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:31.632 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:31 smithi086 ceph-mon[122131]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:12:31.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:31 smithi123 ceph-mon[122926]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:31.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:31 smithi123 ceph-mon[122926]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-07T23:12:31.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:31 smithi123 ceph-mon[122926]: osdmap e44: 7 total, 6 up, 7 in 2023-10-07T23:12:31.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:31.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:31 smithi123 ceph-mon[122926]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:12:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-07T23:12:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-07T23:12:32.175 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2023-10-07T23:12:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2023-10-07T23:12:32.175 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127029 s, 825 MB/s 2023-10-07T23:12:32.175 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2023-10-07T23:12:32.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:32 smithi067 ceph-mon[82723]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:12:33.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:32 smithi067 ceph-mon[82723]: osdmap e45: 7 total, 6 up, 7 in 2023-10-07T23:12:33.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:32 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.368 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:32 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:32 smithi086 ceph-mon[122131]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:12:33.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:32 smithi086 ceph-mon[122131]: osdmap e45: 7 total, 6 up, 7 in 2023-10-07T23:12:33.369 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:32 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.658 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:32 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.658 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:32 smithi123 ceph-mon[122926]: from='osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:12:33.658 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:32 smithi123 ceph-mon[122926]: osdmap e45: 7 total, 6 up, 7 in 2023-10-07T23:12:33.659 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:32 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.659 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:32 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2023-10-07T23:12:33.661 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:12:33.661 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:12:33.661 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:33.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] boot 2023-10-07T23:12:33.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: osdmap e46: 7 total, 7 up, 7 in 2023-10-07T23:12:33.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:33 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:12:33.663 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] boot 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: osdmap e46: 7 total, 7 up, 7 in 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:33.664 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:33.665 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:33 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.693 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:12:33.725 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:12:33.725 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2023-10-07T23:12:33.725 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.725 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: osd.6 [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] boot 2023-10-07T23:12:33.725 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: osdmap e46: 7 total, 7 up, 7 in 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:33.726 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:33 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:34.255 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:12:34.521 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:34 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi067 2023-10-07T23:12:34.521 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:34 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi067 to 5755M 2023-10-07T23:12:34.521 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:34 smithi086 ceph-mon[122131]: osdmap e47: 7 total, 7 up, 7 in 2023-10-07T23:12:34.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:34 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi067 2023-10-07T23:12:34.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:34 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi067 to 5755M 2023-10-07T23:12:34.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:34 smithi123 ceph-mon[122926]: osdmap e47: 7 total, 7 up, 7 in 2023-10-07T23:12:34.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:34 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi067 2023-10-07T23:12:34.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:34 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi067 to 5755M 2023-10-07T23:12:34.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:34 smithi067 ceph-mon[82723]: osdmap e47: 7 total, 7 up, 7 in 2023-10-07T23:12:35.542 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:35 smithi086 ceph-mon[122131]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:35 smithi123 ceph-mon[122926]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 -- 172.21.15.86:0/3410645070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fe980 msgr2=0x7f0ab41052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 --2- 172.21.15.86:0/3410645070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fe980 0x7f0ab41052b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0049c0 tx=0x7f0aac009e30 comp rx=0 tx=0).stop 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 -- 172.21.15.86:0/3410645070 shutdown_connections 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 --2- 172.21.15.86:0/3410645070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fe980 0x7f0ab41052b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 --2- 172.21.15.86:0/3410645070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ab40fce60 0x7f0ab40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 --2- 172.21.15.86:0/3410645070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 -- 172.21.15.86:0/3410645070 >> 172.21.15.86:0/3410645070 conn(0x7f0ab40f85a0 msgr2=0x7f0ab40fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 -- 172.21.15.86:0/3410645070 shutdown_connections 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.742+0000 7f0abc656700 1 -- 172.21.15.86:0/3410645070 wait complete. 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 Processor -- start 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 -- start start 2023-10-07T23:12:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ab40fce60 0x7f0ab410d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab40fe980 0x7f0ab4114db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41153c0 con 0x7f0ab40fce60 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4115530 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0abc656700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41156f0 con 0x7f0ab40fe980 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab410dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47876/0 (socket says 172.21.15.86:47876) 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 -- 172.21.15.86:0/2121349559 learned_addr learned my addr 172.21.15.86:0/2121349559 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ababf3700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab40fe980 0x7f0ab4114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 -- 172.21.15.86:0/2121349559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab40fe980 msgr2=0x7f0ab4114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab40fe980 0x7f0ab4114db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 -- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ab40fce60 msgr2=0x7f0ab410d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ab40fce60 0x7f0ab410d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.743+0000 7f0ab9bf1700 1 -- 172.21.15.86:0/2121349559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac005040 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.744+0000 7f0ab9bf1700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab410dd20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800ebf0 tx=0x7f0aa800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.744+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8007cc0 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.744+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab4115970 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.744+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0aa800f040 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.744+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80094c0 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.744+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab4115e90 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.745+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4103a00 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.746+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f0aa800c680 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.746+0000 7f0ab2ffd700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a9c09b770 0x7f0a9c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:12:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.746+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4961+0+0 (secure 0 0 0) 0x7f0aa80c3f70 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.747+0000 7f0aba3f2700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a9c09b770 0x7f0a9c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:12:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.747+0000 7f0aba3f2700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a9c09b770 0x7f0a9c09dc20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007fd0 tx=0x7f0aa4007bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:12:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.751+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f0aa8090490 con 0x7f0ab40fdfd0 2023-10-07T23:12:35.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:35 smithi067 ceph-mon[82723]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:35.938+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ab405e080 con 0x7f0a9c09b770 2023-10-07T23:12:36.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:12:36.654 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:12:36.654 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:36.689 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:12:36.689 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:12:36.689 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:36.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:12:36.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:12:36.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:37 smithi086 ceph-mon[122131]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:37 smithi086 ceph-mon[122131]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:12:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:37 smithi123 ceph-mon[122926]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:37 smithi123 ceph-mon[122926]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:12:37.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:37 smithi067 ceph-mon[82723]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:37.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:37 smithi067 ceph-mon[82723]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:12:39.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/2422963413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]: dispatch 2023-10-07T23:12:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]: dispatch 2023-10-07T23:12:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]': finished 2023-10-07T23:12:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: osdmap e48: 8 total, 7 up, 8 in 2023-10-07T23:12:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:39 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/807866171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:12:39.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/2422963413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]: dispatch 2023-10-07T23:12:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]: dispatch 2023-10-07T23:12:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]': finished 2023-10-07T23:12:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: osdmap e48: 8 total, 7 up, 8 in 2023-10-07T23:12:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:39.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:39 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/807866171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:12:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/2422963413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]: dispatch 2023-10-07T23:12:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]: dispatch 2023-10-07T23:12:39.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab847d18-8863-428b-9647-eadc654dc957"}]': finished 2023-10-07T23:12:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: osdmap e48: 8 total, 7 up, 8 in 2023-10-07T23:12:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:39.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:39 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/807866171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:12:41.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:41 smithi123 ceph-mon[122926]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:41 smithi086 ceph-mon[122131]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:41.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:41 smithi067 ceph-mon[82723]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:43.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:43 smithi123 ceph-mon[122926]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:43.688 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:43 smithi086 ceph-mon[122131]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:43.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:43 smithi067 ceph-mon[82723]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:44.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:44 smithi086 ceph-mon[122131]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:45.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:44 smithi123 ceph-mon[122926]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:44 smithi067 ceph-mon[82723]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:46.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:45 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-07T23:12:46.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:45 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:45 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-07T23:12:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:45 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:46.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:45 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-07T23:12:46.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:45 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:47.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:46 smithi123 ceph-mon[122926]: Deploying daemon osd.7 on smithi086 2023-10-07T23:12:47.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:46 smithi123 ceph-mon[122926]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:46 smithi086 ceph-mon[122131]: Deploying daemon osd.7 on smithi086 2023-10-07T23:12:47.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:46 smithi086 ceph-mon[122131]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:46 smithi067 ceph-mon[82723]: Deploying daemon osd.7 on smithi086 2023-10-07T23:12:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:46 smithi067 ceph-mon[82723]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:49.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:48 smithi086 ceph-mon[122131]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:48 smithi067 ceph-mon[82723]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:48 smithi123 ceph-mon[122926]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:51.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:50 smithi086 ceph-mon[122131]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:50 smithi067 ceph-mon[82723]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:50 smithi123 ceph-mon[122926]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:52.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:52.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:52.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:52.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:52.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:52.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:52.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:52.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:52.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:53.196 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:52 smithi086 ceph-mon[122131]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:52 smithi067 ceph-mon[82723]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:53.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:52 smithi123 ceph-mon[122926]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:54.461 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 7 on host 'smithi086' 2023-10-07T23:12:54.462 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.461+0000 7f0ab2ffd700 1 -- 172.21.15.86:0/2121349559 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0ab405e080 con 0x7f0a9c09b770 2023-10-07T23:12:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a9c09b770 msgr2=0x7f0a9c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a9c09b770 0x7f0a9c09dc20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007fd0 tx=0x7f0aa4007bb0 comp rx=0 tx=0).stop 2023-10-07T23:12:54.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 msgr2=0x7f0ab410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab410dd20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800ebf0 tx=0x7f0aa800b040 comp rx=0 tx=0).stop 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 shutdown_connections 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a9c09b770 0x7f0a9c09dc20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab40fe980 0x7f0ab4114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ab40fce60 0x7f0ab410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 --2- 172.21.15.86:0/2121349559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab40fdfd0 0x7f0ab410dd20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 >> 172.21.15.86:0/2121349559 conn(0x7f0ab40f85a0 msgr2=0x7f0ab40fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 shutdown_connections 2023-10-07T23:12:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:12:54.463+0000 7f0abc656700 1 -- 172.21.15.86:0/2121349559 wait complete. 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:54.987 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:55.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:55.107 DEBUG:teuthology.orchestra.run.smithi086:osd.7> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-07T23:12:55.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:55.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:12:55.150 INFO:tasks.cephadm:Deploying osd.8 on smithi123 with /dev/vg_nvme/lv_2... 2023-10-07T23:12:55.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-07T23:12:55.180 INFO:journalctl@ceph.osd.7.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:12:55.738 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:12:56.044 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:55 smithi123 ceph-mon[122926]: from='osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-07T23:12:56.044 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:55 smithi123 ceph-mon[122926]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-07T23:12:56.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:55 smithi086 ceph-mon[122131]: from='osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-07T23:12:56.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:55 smithi086 ceph-mon[122131]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-07T23:12:56.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:55 smithi067 ceph-mon[82723]: from='osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-07T23:12:56.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:55 smithi067 ceph-mon[82723]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-07T23:12:57.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:56 smithi123 ceph-mon[122926]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-07T23:12:57.115 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:56 smithi123 ceph-mon[122926]: osdmap e49: 8 total, 7 up, 8 in 2023-10-07T23:12:57.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:57.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:56 smithi123 ceph-mon[122926]: from='osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:12:57.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:56 smithi123 ceph-mon[122926]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:12:57.116 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:56 smithi123 ceph-mon[122926]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:57.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:56 smithi086 ceph-mon[122131]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-07T23:12:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:56 smithi086 ceph-mon[122131]: osdmap e49: 8 total, 7 up, 8 in 2023-10-07T23:12:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:56 smithi086 ceph-mon[122131]: from='osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:12:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:56 smithi086 ceph-mon[122131]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:12:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:56 smithi086 ceph-mon[122131]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:57.198 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:12:56 smithi086 conmon[148156]: 2023-10-07T23:12:56.807+0000 7efddf662700 -1 osd.7 0 waiting for initial osdmap 2023-10-07T23:12:57.199 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:12:56 smithi086 conmon[148156]: 2023-10-07T23:12:56.816+0000 7efddbc60700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:12:57.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:56 smithi067 ceph-mon[82723]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-07T23:12:57.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:56 smithi067 ceph-mon[82723]: osdmap e49: 8 total, 7 up, 8 in 2023-10-07T23:12:57.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:57.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:56 smithi067 ceph-mon[82723]: from='osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:12:57.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:56 smithi067 ceph-mon[82723]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:12:57.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:56 smithi067 ceph-mon[82723]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2023-10-07T23:12:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-07T23:12:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-07T23:12:57.631 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-10-07T23:12:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-10-07T23:12:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113716 s, 922 MB/s 2023-10-07T23:12:57.632 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-10-07T23:12:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:57 smithi086 ceph-mon[122131]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:12:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:57 smithi086 ceph-mon[122131]: osdmap e50: 8 total, 7 up, 8 in 2023-10-07T23:12:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:57 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:57 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:58.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:57 smithi123 ceph-mon[122926]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:12:58.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:57 smithi123 ceph-mon[122926]: osdmap e50: 8 total, 7 up, 8 in 2023-10-07T23:12:58.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:57 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:58.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:57 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:58.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:57 smithi067 ceph-mon[82723]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:12:58.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:57 smithi067 ceph-mon[82723]: osdmap e50: 8 total, 7 up, 8 in 2023-10-07T23:12:58.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:57 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:58.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:57 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:58.269 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2023-10-07T23:12:58.850 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] boot 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: osdmap e51: 8 total, 8 up, 8 in 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi086 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi086 to 5724M 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:12:58 smithi123 ceph-mon[122926]: osdmap e52: 8 total, 8 up, 8 in 2023-10-07T23:12:59.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:12:59.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:12:59.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] boot 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: osdmap e51: 8 total, 8 up, 8 in 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi086 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi086 to 5724M 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:12:58 smithi086 ceph-mon[122131]: osdmap e52: 8 total, 8 up, 8 in 2023-10-07T23:12:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:12:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: osd.7 [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] boot 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: osdmap e51: 8 total, 8 up, 8 in 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi086 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi086 to 5724M 2023-10-07T23:12:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:12:59.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:12:59.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:12:59.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:12:58 smithi067 ceph-mon[82723]: osdmap e52: 8 total, 8 up, 8 in 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.325+0000 7f137c44e700 1 -- 172.21.15.123:0/3720238786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fbd50 msgr2=0x7f13740fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.325+0000 7f137c44e700 1 --2- 172.21.15.123:0/3720238786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fbd50 0x7f13740fc160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13700049c0 tx=0x7f1370009e30 comp rx=0 tx=0).stop 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- 172.21.15.123:0/3720238786 shutdown_connections 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 --2- 172.21.15.123:0/3720238786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13740fd8f0 0x7f13741041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 --2- 172.21.15.123:0/3720238786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fbd50 0x7f13740fc160 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 --2- 172.21.15.123:0/3720238786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fcf60 0x7f13740fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- 172.21.15.123:0/3720238786 >> 172.21.15.123:0/3720238786 conn(0x7f13740f74b0 msgr2=0x7f13740f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:13:00.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- 172.21.15.123:0/3720238786 shutdown_connections 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- 172.21.15.123:0/3720238786 wait complete. 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 Processor -- start 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- start start 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13740fbd50 0x7f137410cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 0x7f137410d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fd8f0 0x7f13741141f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374114830 con 0x7f13740fd8f0 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13741149a0 con 0x7f13740fbd50 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f137c44e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374114b60 con 0x7f13740fcf60 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f13799e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 0x7f137410d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f13799e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 0x7f137410d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37860/0 (socket says 172.21.15.123:37860) 2023-10-07T23:13:00.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.326+0000 7f13799e9700 1 -- 172.21.15.123:0/437892187 learned_addr learned my addr 172.21.15.123:0/437892187 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f137a1ea700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13740fbd50 0x7f137410cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f13799e9700 1 -- 172.21.15.123:0/437892187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13740fbd50 msgr2=0x7f137410cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f13799e9700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13740fbd50 0x7f137410cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f13799e9700 1 -- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fd8f0 msgr2=0x7f13741141f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f137a9eb700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fd8f0 0x7f13741141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f13799e9700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fd8f0 0x7f13741141f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f13799e9700 1 -- 172.21.15.123:0/437892187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1370005040 con 0x7f13740fcf60 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f137a9eb700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fd8f0 0x7f13741141f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f13799e9700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 0x7f137410d1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f136400b810 tx=0x7f1364009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136400f5f0 con 0x7f13740fcf60 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136400fc20 con 0x7f13740fcf60 2023-10-07T23:13:00.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1374114e40 con 0x7f13740fcf60 2023-10-07T23:13:00.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364010c80 con 0x7f13740fcf60 2023-10-07T23:13:00.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.327+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13741152a0 con 0x7f13740fcf60 2023-10-07T23:13:00.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.328+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374103420 con 0x7f13740fcf60 2023-10-07T23:13:00.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.329+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f136400fd90 con 0x7f13740fcf60 2023-10-07T23:13:00.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.330+0000 7f136affd700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f136009b7f0 0x7f136009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:00.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.330+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5422+0+0 (secure 0 0 0) 0x7f13640c4c00 con 0x7f13740fcf60 2023-10-07T23:13:00.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.330+0000 7f137a1ea700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f136009b7f0 0x7f136009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:00.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.331+0000 7f137a1ea700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f136009b7f0 0x7f136009dca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1370009fd0 tx=0x7f1370009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:13:00.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.333+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f1364090fe0 con 0x7f13740fcf60 2023-10-07T23:13:00.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:00.523+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f137402b420 con 0x7f136009b7f0 2023-10-07T23:13:01.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:00 smithi123 ceph-mon[122926]: osdmap e53: 8 total, 8 up, 8 in 2023-10-07T23:13:01.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:00 smithi123 ceph-mon[122926]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:01.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:01.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:00 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:01.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:00 smithi086 ceph-mon[122131]: osdmap e53: 8 total, 8 up, 8 in 2023-10-07T23:13:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:00 smithi086 ceph-mon[122131]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:00 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:01.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:00 smithi067 ceph-mon[82723]: osdmap e53: 8 total, 8 up, 8 in 2023-10-07T23:13:01.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:00 smithi067 ceph-mon[82723]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:01.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:01.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:01.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:00 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:02.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:01 smithi123 ceph-mon[122926]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:02.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:01 smithi123 ceph-mon[122926]: osdmap e54: 8 total, 8 up, 8 in 2023-10-07T23:13:02.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:01 smithi086 ceph-mon[122131]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:02.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:01 smithi086 ceph-mon[122131]: osdmap e54: 8 total, 8 up, 8 in 2023-10-07T23:13:02.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:01 smithi067 ceph-mon[82723]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:02.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:01 smithi067 ceph-mon[82723]: osdmap e54: 8 total, 8 up, 8 in 2023-10-07T23:13:03.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:02 smithi123 ceph-mon[122926]: osdmap e55: 8 total, 8 up, 8 in 2023-10-07T23:13:03.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:02 smithi123 ceph-mon[122926]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:03.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:02 smithi086 ceph-mon[122131]: osdmap e55: 8 total, 8 up, 8 in 2023-10-07T23:13:03.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:02 smithi086 ceph-mon[122131]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:03.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:02 smithi067 ceph-mon[82723]: osdmap e55: 8 total, 8 up, 8 in 2023-10-07T23:13:03.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:02 smithi067 ceph-mon[82723]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:04.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:03 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/1264801502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]: dispatch 2023-10-07T23:13:04.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:03 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]: dispatch 2023-10-07T23:13:04.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:03 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]': finished 2023-10-07T23:13:04.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:03 smithi123 ceph-mon[122926]: osdmap e56: 9 total, 8 up, 9 in 2023-10-07T23:13:04.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:03 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:03 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/1264801502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]: dispatch 2023-10-07T23:13:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:03 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]: dispatch 2023-10-07T23:13:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:03 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]': finished 2023-10-07T23:13:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:03 smithi086 ceph-mon[122131]: osdmap e56: 9 total, 8 up, 9 in 2023-10-07T23:13:04.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:03 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:03 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/1264801502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]: dispatch 2023-10-07T23:13:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:03 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]: dispatch 2023-10-07T23:13:04.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:03 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e766f43b-63d1-4a5c-a427-2caca15c2af6"}]': finished 2023-10-07T23:13:04.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:03 smithi067 ceph-mon[82723]: osdmap e56: 9 total, 8 up, 9 in 2023-10-07T23:13:04.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:03 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:05.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:04 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/1448480400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:05.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:04 smithi123 ceph-mon[122926]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:04 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/1448480400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:04 smithi086 ceph-mon[122131]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:04 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/1448480400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:04 smithi067 ceph-mon[82723]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:07.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:06 smithi123 ceph-mon[122926]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:06 smithi086 ceph-mon[122131]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:06 smithi067 ceph-mon[82723]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:09.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:08 smithi123 ceph-mon[122926]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:09.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:08 smithi067 ceph-mon[82723]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:09.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:08 smithi086 ceph-mon[122131]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:11.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:10 smithi123 ceph-mon[122926]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:11.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-07T23:13:11.229 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:10 smithi067 ceph-mon[82723]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-07T23:13:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:11.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:10 smithi086 ceph-mon[122131]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:11.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-07T23:13:11.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:12.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:11 smithi067 ceph-mon[82723]: Deploying daemon osd.8 on smithi123 2023-10-07T23:13:12.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:11 smithi123 ceph-mon[122926]: Deploying daemon osd.8 on smithi123 2023-10-07T23:13:12.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:11 smithi086 ceph-mon[122131]: Deploying daemon osd.8 on smithi123 2023-10-07T23:13:13.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:12 smithi067 ceph-mon[82723]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:13.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:12 smithi123 ceph-mon[122926]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:13.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:12 smithi086 ceph-mon[122131]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:14 smithi123 ceph-mon[122926]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:14 smithi086 ceph-mon[122131]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:15.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:14 smithi067 ceph-mon[82723]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:17.059 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:16 smithi123 ceph-mon[122926]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:17.059 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:17.060 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:17.060 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:17.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:16 smithi067 ceph-mon[82723]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:17.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:17.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:17.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:16 smithi086 ceph-mon[122131]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:18 smithi067 ceph-mon[82723]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.297+0000 7f136affd700 1 -- 172.21.15.123:0/437892187 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f137402b420 con 0x7f136009b7f0 2023-10-07T23:13:19.299 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 8 on host 'smithi123' 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f136009b7f0 msgr2=0x7f136009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f136009b7f0 0x7f136009dca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1370009fd0 tx=0x7f1370009e30 comp rx=0 tx=0).stop 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 msgr2=0x7f137410d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 0x7f137410d1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f136400b810 tx=0x7f1364009250 comp rx=0 tx=0).stop 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 shutdown_connections 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13740fbd50 0x7f137410cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f136009b7f0 0x7f136009dca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13740fd8f0 0x7f13741141f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 --2- 172.21.15.123:0/437892187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13740fcf60 0x7f137410d1a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 >> 172.21.15.123:0/437892187 conn(0x7f13740f74b0 msgr2=0x7f13741005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:13:19.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 shutdown_connections 2023-10-07T23:13:19.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:13:19.299+0000 7f137c44e700 1 -- 172.21.15.123:0/437892187 wait complete. 2023-10-07T23:13:19.321 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:18 smithi123 ceph-mon[122926]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:19.321 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.321 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.321 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:19.321 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:19.322 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:18 smithi086 ceph-mon[122131]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:19.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:19.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.005 DEBUG:teuthology.orchestra.run.smithi123:osd.8> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-07T23:13:20.007 INFO:tasks.cephadm:Deploying osd.9 on smithi067 with /dev/vg_nvme/lv_1... 2023-10-07T23:13:20.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-07T23:13:20.035 INFO:journalctl@ceph.osd.8.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:13:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:20.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:20 smithi067 ceph-mon[82723]: from='osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-07T23:13:20.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:20 smithi067 ceph-mon[82723]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-07T23:13:20.525 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.525 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.525 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:20.525 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:20 smithi123 ceph-mon[122926]: from='osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-07T23:13:20.526 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:20 smithi123 ceph-mon[122926]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-07T23:13:20.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:20.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:20.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:20 smithi086 ceph-mon[122131]: from='osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-07T23:13:20.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:20 smithi086 ceph-mon[122131]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-07T23:13:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:13:21.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:21 smithi067 ceph-mon[82723]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:21.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:21 smithi067 ceph-mon[82723]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-07T23:13:21.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:21 smithi067 ceph-mon[82723]: osdmap e57: 9 total, 8 up, 9 in 2023-10-07T23:13:21.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:21.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:21 smithi067 ceph-mon[82723]: from='osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:13:21.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:21 smithi067 ceph-mon[82723]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:13:21.503 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:13:21 smithi123 conmon[150509]: 2023-10-07T23:13:21.266+0000 7f05b5897700 -1 osd.8 0 waiting for initial osdmap 2023-10-07T23:13:21.503 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:13:21 smithi123 conmon[150509]: 2023-10-07T23:13:21.277+0000 7f05aee8f700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:13:21.503 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:21 smithi123 ceph-mon[122926]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:21.503 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:21 smithi123 ceph-mon[122926]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-07T23:13:21.503 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:21 smithi123 ceph-mon[122926]: osdmap e57: 9 total, 8 up, 9 in 2023-10-07T23:13:21.504 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:21.504 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:21 smithi123 ceph-mon[122926]: from='osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:13:21.504 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:21 smithi123 ceph-mon[122926]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:13:21.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:21 smithi086 ceph-mon[122131]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:21 smithi086 ceph-mon[122131]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-07T23:13:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:21 smithi086 ceph-mon[122131]: osdmap e57: 9 total, 8 up, 9 in 2023-10-07T23:13:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:21 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:21 smithi086 ceph-mon[122131]: from='osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:13:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:21 smithi086 ceph-mon[122131]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:13:22.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:22 smithi123 ceph-mon[122926]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:13:22.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:22 smithi123 ceph-mon[122926]: osdmap e58: 9 total, 8 up, 9 in 2023-10-07T23:13:22.536 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:22.537 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:22.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:22 smithi086 ceph-mon[122131]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:13:22.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:22 smithi086 ceph-mon[122131]: osdmap e58: 9 total, 8 up, 9 in 2023-10-07T23:13:22.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:22.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:22.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:22 smithi067 ceph-mon[82723]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:13:22.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:22 smithi067 ceph-mon[82723]: osdmap e58: 9 total, 8 up, 9 in 2023-10-07T23:13:22.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:22.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:22.829 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-07T23:13:22.829 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-07T23:13:22.830 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2023-10-07T23:13:22.830 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2023-10-07T23:13:22.830 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0317371 s, 330 MB/s 2023-10-07T23:13:22.830 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2023-10-07T23:13:23.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2023-10-07T23:13:23.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] boot 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: osdmap e59: 9 total, 9 up, 9 in 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:23.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:23.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] boot 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: osdmap e59: 9 total, 9 up, 9 in 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:23.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:23.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: osd.8 [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] boot 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: osdmap e59: 9 total, 9 up, 9 in 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:23.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:23.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:23.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:13:24.478 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:24 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi123 2023-10-07T23:13:24.479 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:24 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi123 to 5724M 2023-10-07T23:13:24.479 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:24 smithi067 ceph-mon[82723]: osdmap e60: 9 total, 9 up, 9 in 2023-10-07T23:13:24.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:24 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi123 2023-10-07T23:13:24.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:24 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi123 to 5724M 2023-10-07T23:13:24.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:24 smithi123 ceph-mon[122926]: osdmap e60: 9 total, 9 up, 9 in 2023-10-07T23:13:24.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:24 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi123 2023-10-07T23:13:24.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:24 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi123 to 5724M 2023-10-07T23:13:24.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:24 smithi086 ceph-mon[122131]: osdmap e60: 9 total, 9 up, 9 in 2023-10-07T23:13:25.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 -- 172.21.15.67:0/677822219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73981002b0 msgr2=0x7f7398102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:25.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 --2- 172.21.15.67:0/677822219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73981002b0 0x7f7398102690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73840049c0 tx=0x7f7384009e30 comp rx=0 tx=0).stop 2023-10-07T23:13:25.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 -- 172.21.15.67:0/677822219 shutdown_connections 2023-10-07T23:13:25.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 --2- 172.21.15.67:0/677822219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73980fd990 0x7f73980ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 --2- 172.21.15.67:0/677822219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7398102bd0 0x7f7398104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 --2- 172.21.15.67:0/677822219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73981002b0 0x7f7398102690 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 -- 172.21.15.67:0/677822219 >> 172.21.15.67:0/677822219 conn(0x7f73980f74b0 msgr2=0x7f73980f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 -- 172.21.15.67:0/677822219 shutdown_connections 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.346+0000 7f739ec12700 1 -- 172.21.15.67:0/677822219 wait complete. 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 Processor -- start 2023-10-07T23:13:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 -- start start 2023-10-07T23:13:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 0x7f739810fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73981002b0 0x7f7398110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398102bd0 0x7f7398117310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398117920 con 0x7f7398102bd0 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398117a90 con 0x7f73981002b0 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739ec12700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398117c50 con 0x7f73980fd990 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739c9ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 0x7f739810fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.347+0000 7f739d1af700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398102bd0 0x7f7398117310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f739c9ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 0x7f739810fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:53078/0 (socket says 172.21.15.67:53078) 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73981002b0 0x7f7398110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f739c9ae700 1 -- 172.21.15.67:0/2911081985 learned_addr learned my addr 172.21.15.67:0/2911081985 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:13:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 -- 172.21.15.67:0/2911081985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 msgr2=0x7f739810fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 0x7f739810fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 -- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398102bd0 msgr2=0x7f7398117310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398102bd0 0x7f7398117310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 -- 172.21.15.67:0/2911081985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384005040 con 0x7f73981002b0 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f739c9ae700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 0x7f739810fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f739d1af700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398102bd0 0x7f7398117310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f7397fff700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73981002b0 0x7f7398110280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73840049c0 tx=0x7f7384009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400e070 con 0x7f73981002b0 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7384004160 con 0x7f73981002b0 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7398117ed0 con 0x7f73981002b0 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.348+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73840123c0 con 0x7f73981002b0 2023-10-07T23:13:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.349+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7398118350 con 0x7f73981002b0 2023-10-07T23:13:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.350+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7398104700 con 0x7f73981002b0 2023-10-07T23:13:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.352+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f738401c610 con 0x7f73981002b0 2023-10-07T23:13:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.353+0000 7f73957fa700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f738809b770 0x7f738809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.353+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f73840c5e90 con 0x7f73981002b0 2023-10-07T23:13:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.353+0000 7f739c9ae700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f738809b770 0x7f738809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.354+0000 7f739c9ae700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f738809b770 0x7f738809dc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f738c0041d0 tx=0x7f738c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:13:25.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.355+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7384091fb0 con 0x7f73981002b0 2023-10-07T23:13:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:25 smithi067 ceph-mon[82723]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:25 smithi067 ceph-mon[82723]: osdmap e61: 9 total, 9 up, 9 in 2023-10-07T23:13:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:25.568+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f739802b360 con 0x7f738809b770 2023-10-07T23:13:25.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:25 smithi123 ceph-mon[122926]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:25 smithi123 ceph-mon[122926]: osdmap e61: 9 total, 9 up, 9 in 2023-10-07T23:13:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:25 smithi086 ceph-mon[122131]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:25 smithi086 ceph-mon[122131]: osdmap e61: 9 total, 9 up, 9 in 2023-10-07T23:13:26.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:26 smithi067 ceph-mon[82723]: osdmap e62: 9 total, 9 up, 9 in 2023-10-07T23:13:26.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:26.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:26.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:26.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:26 smithi123 ceph-mon[122926]: osdmap e62: 9 total, 9 up, 9 in 2023-10-07T23:13:26.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:26.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:26.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:26.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:26 smithi086 ceph-mon[122131]: osdmap e62: 9 total, 9 up, 9 in 2023-10-07T23:13:26.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:26.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:26.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:27.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:27 smithi067 ceph-mon[82723]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:27.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:27 smithi067 ceph-mon[82723]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:27.559 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:27 smithi067 ceph-mon[82723]: osdmap e63: 9 total, 9 up, 9 in 2023-10-07T23:13:27.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:27 smithi123 ceph-mon[122926]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:27.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:27 smithi123 ceph-mon[122926]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:27.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:27 smithi123 ceph-mon[122926]: osdmap e63: 9 total, 9 up, 9 in 2023-10-07T23:13:27.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:27 smithi086 ceph-mon[122131]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:27.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:27 smithi086 ceph-mon[122131]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:27.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:27 smithi086 ceph-mon[122131]: osdmap e63: 9 total, 9 up, 9 in 2023-10-07T23:13:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:29 smithi123 ceph-mon[122926]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-07T23:13:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:29 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2999504463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3781cb60-d57e-40f1-a762-5ddfbfa66a15"}]: dispatch 2023-10-07T23:13:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:29 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2999504463' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3781cb60-d57e-40f1-a762-5ddfbfa66a15"}]': finished 2023-10-07T23:13:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:29 smithi123 ceph-mon[122926]: osdmap e64: 10 total, 9 up, 10 in 2023-10-07T23:13:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:29 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3343277022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:29 smithi086 ceph-mon[122131]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-07T23:13:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:29 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2999504463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3781cb60-d57e-40f1-a762-5ddfbfa66a15"}]: dispatch 2023-10-07T23:13:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:29 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2999504463' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3781cb60-d57e-40f1-a762-5ddfbfa66a15"}]': finished 2023-10-07T23:13:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:29 smithi086 ceph-mon[122131]: osdmap e64: 10 total, 9 up, 10 in 2023-10-07T23:13:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:29 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3343277022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:29 smithi067 ceph-mon[82723]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-07T23:13:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:29 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2999504463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3781cb60-d57e-40f1-a762-5ddfbfa66a15"}]: dispatch 2023-10-07T23:13:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:29 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2999504463' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3781cb60-d57e-40f1-a762-5ddfbfa66a15"}]': finished 2023-10-07T23:13:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:29 smithi067 ceph-mon[82723]: osdmap e64: 10 total, 9 up, 10 in 2023-10-07T23:13:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:29 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3343277022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:31.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:31 smithi123 ceph-mon[122926]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-07T23:13:31.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:31 smithi086 ceph-mon[122131]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-07T23:13:31.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:31 smithi067 ceph-mon[82723]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-07T23:13:33.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:33 smithi123 ceph-mon[122926]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-10-07T23:13:33.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:33 smithi086 ceph-mon[122131]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-10-07T23:13:33.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:33 smithi067 ceph-mon[82723]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-10-07T23:13:35.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:34 smithi067 ceph-mon[82723]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-07T23:13:35.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:34 smithi123 ceph-mon[122926]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-07T23:13:35.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:34 smithi086 ceph-mon[122131]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-07T23:13:36.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:35 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-07T23:13:36.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:35 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:36.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:35 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-07T23:13:36.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:35 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:36.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:35 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-07T23:13:36.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:35 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:36 smithi067 ceph-mon[82723]: Deploying daemon osd.9 on smithi067 2023-10-07T23:13:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:36 smithi067 ceph-mon[82723]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-07T23:13:37.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:36 smithi123 ceph-mon[122926]: Deploying daemon osd.9 on smithi067 2023-10-07T23:13:37.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:36 smithi123 ceph-mon[122926]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-07T23:13:37.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:36 smithi086 ceph-mon[122131]: Deploying daemon osd.9 on smithi067 2023-10-07T23:13:37.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:36 smithi086 ceph-mon[122131]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-07T23:13:39.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:38 smithi067 ceph-mon[82723]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:39.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:38 smithi123 ceph-mon[122926]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:39.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:38 smithi086 ceph-mon[122131]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:41.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:40 smithi067 ceph-mon[82723]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:41.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:40 smithi123 ceph-mon[122926]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:41.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:40 smithi086 ceph-mon[122131]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:43.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:43.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:43.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:42 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:43.051 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:42 smithi067 ceph-mon[82723]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:43.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:43.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:43.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:42 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:43.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:42 smithi123 ceph-mon[122926]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:43.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:43.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:42 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:42 smithi086 ceph-mon[122131]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:45.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:44 smithi067 ceph-mon[82723]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:45.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:44 smithi123 ceph-mon[122926]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:45.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:44 smithi086 ceph-mon[122131]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:45.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.375+0000 7f73957fa700 1 -- 172.21.15.67:0/2911081985 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f739802b360 con 0x7f738809b770 2023-10-07T23:13:45.376 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 9 on host 'smithi067' 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f738809b770 msgr2=0x7f738809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f738809b770 0x7f738809dc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f738c0041d0 tx=0x7f738c00b040 comp rx=0 tx=0).stop 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73981002b0 msgr2=0x7f7398110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73981002b0 0x7f7398110280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73840049c0 tx=0x7f7384009e30 comp rx=0 tx=0).stop 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 shutdown_connections 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f738809b770 0x7f738809dc20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398102bd0 0x7f7398117310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73980fd990 0x7f739810fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 --2- 172.21.15.67:0/2911081985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73981002b0 0x7f7398110280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 >> 172.21.15.67:0/2911081985 conn(0x7f73980f74b0 msgr2=0x7f73980fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:13:45.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 shutdown_connections 2023-10-07T23:13:45.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:13:45.378+0000 7f739ec12700 1 -- 172.21.15.67:0/2911081985 wait complete. 2023-10-07T23:13:45.830 DEBUG:teuthology.orchestra.run.smithi067:osd.9> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-07T23:13:45.835 INFO:tasks.cephadm:Deploying osd.10 on smithi086 with /dev/vg_nvme/lv_1... 2023-10-07T23:13:45.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-07T23:13:45.901 INFO:journalctl@ceph.osd.9.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:13:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-07T23:13:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:46 smithi086 ceph-mon[122131]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-07T23:13:46.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-07T23:13:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:46 smithi123 ceph-mon[122926]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-07T23:13:46.425 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:13:46.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-07T23:13:46.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:46 smithi067 ceph-mon[82723]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-07T23:13:47.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:47 smithi123 ceph-mon[122926]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:47.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:47 smithi123 ceph-mon[122926]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-07T23:13:47.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:47 smithi123 ceph-mon[122926]: osdmap e65: 10 total, 9 up, 10 in 2023-10-07T23:13:47.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:47.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:47 smithi123 ceph-mon[122926]: from='osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:13:47.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:47 smithi123 ceph-mon[122926]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:13:47.672 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:47 smithi086 ceph-mon[122131]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:47.672 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:47 smithi086 ceph-mon[122131]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-07T23:13:47.672 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:47 smithi086 ceph-mon[122131]: osdmap e65: 10 total, 9 up, 10 in 2023-10-07T23:13:47.672 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:47.672 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:47 smithi086 ceph-mon[122131]: from='osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:13:47.673 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:47 smithi086 ceph-mon[122131]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:13:47.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:47 smithi067 ceph-mon[82723]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:47.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:47 smithi067 ceph-mon[82723]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-07T23:13:47.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:47 smithi067 ceph-mon[82723]: osdmap e65: 10 total, 9 up, 10 in 2023-10-07T23:13:47.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:47.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:47 smithi067 ceph-mon[82723]: from='osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:13:47.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:47 smithi067 ceph-mon[82723]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2023-10-07T23:13:47.755 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:13:47 smithi067 conmon[130383]: 2023-10-07T23:13:47.369+0000 7f2148248700 -1 osd.9 0 waiting for initial osdmap 2023-10-07T23:13:47.755 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:13:47 smithi067 conmon[130383]: 2023-10-07T23:13:47.387+0000 7f214103f700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:13:48.226 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-07T23:13:48.226 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-07T23:13:48.226 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2023-10-07T23:13:48.226 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2023-10-07T23:13:48.226 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112273 s, 934 MB/s 2023-10-07T23:13:48.226 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2023-10-07T23:13:48.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:48 smithi123 ceph-mon[122926]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:13:48.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:48 smithi123 ceph-mon[122926]: osdmap e66: 10 total, 9 up, 10 in 2023-10-07T23:13:48.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:48 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:48.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:48 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:48.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:48 smithi067 ceph-mon[82723]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:13:48.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:48 smithi067 ceph-mon[82723]: osdmap e66: 10 total, 9 up, 10 in 2023-10-07T23:13:48.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:48.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:48 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:48.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:48 smithi086 ceph-mon[122131]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2023-10-07T23:13:48.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:48 smithi086 ceph-mon[122131]: osdmap e66: 10 total, 9 up, 10 in 2023-10-07T23:13:48.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:48 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:48.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:48 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:48.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2023-10-07T23:13:49.435 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-07T23:13:49.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:13:49.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:13:49.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:49.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] boot 2023-10-07T23:13:49.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: osdmap e67: 10 total, 10 up, 10 in 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:49.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] boot 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: osdmap e67: 10 total, 10 up, 10 in 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:49.647 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:49.648 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:13:49.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 802 GiB / 805 GiB avail 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: osd.9 [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] boot 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: osdmap e67: 10 total, 10 up, 10 in 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:13:49.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:49.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:49.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:13:49.756 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:13:50.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:50 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi067 2023-10-07T23:13:50.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:50 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi067 to 4316M 2023-10-07T23:13:50.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:50 smithi123 ceph-mon[122926]: osdmap e68: 10 total, 10 up, 10 in 2023-10-07T23:13:50.680 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:50 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi067 2023-10-07T23:13:50.680 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:50 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi067 to 4316M 2023-10-07T23:13:50.680 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:50 smithi086 ceph-mon[122131]: osdmap e68: 10 total, 10 up, 10 in 2023-10-07T23:13:50.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:50 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi067 2023-10-07T23:13:50.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:50 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi067 to 4316M 2023-10-07T23:13:50.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:50 smithi067 ceph-mon[82723]: osdmap e68: 10 total, 10 up, 10 in 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 -- 172.21.15.86:0/2635390576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fbd50 msgr2=0x7f2bc00fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/2635390576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fbd50 0x7f2bc00fc160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00049c0 tx=0x7f2bb0009e30 comp rx=0 tx=0).stop 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 -- 172.21.15.86:0/2635390576 shutdown_connections 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/2635390576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fbd50 0x7f2bc00fc160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/2635390576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc01041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/2635390576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bc00fcf60 0x7f2bc00fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 -- 172.21.15.86:0/2635390576 >> 172.21.15.86:0/2635390576 conn(0x7f2bc00f74d0 msgr2=0x7f2bc00f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 -- 172.21.15.86:0/2635390576 shutdown_connections 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.907+0000 7f2bc63e6700 1 -- 172.21.15.86:0/2635390576 wait complete. 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 Processor -- start 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 -- start start 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc00fbd50 0x7f2bc010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bc00fcf60 0x7f2bc010e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc0115460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0115a70 con 0x7f2bc00fcf60 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0115be0 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc63e6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0115da0 con 0x7f2bc00fbd50 2023-10-07T23:13:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc0115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc0115460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48026/0 (socket says 172.21.15.86:48026) 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 -- 172.21.15.86:0/4244663904 learned_addr learned my addr 172.21.15.86:0/4244663904 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bbffff700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc00fbd50 0x7f2bc010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 -- 172.21.15.86:0/4244663904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc00fbd50 msgr2=0x7f2bc010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc00fbd50 0x7f2bc010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 -- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bc00fcf60 msgr2=0x7f2bc010e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bc00fcf60 0x7f2bc010e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 -- 172.21.15.86:0/4244663904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0005040 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.908+0000 7f2bc4983700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc0115460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800bc30 tx=0x7f2bb8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.909+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80092e0 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.909+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc0116080 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.909+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bb800cc70 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.909+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8015d50 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.909+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc01165c0 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.909+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc0103010 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.910+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f2bb8012070 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.911+0000 7f2bbcff9700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2ba809b870 0x7f2ba809dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:13:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.911+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f2bb80c44d0 con 0x7f2bc00fd8f0 2023-10-07T23:13:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.911+0000 7f2bbffff700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2ba809b870 0x7f2ba809dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:13:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.912+0000 7f2bbffff700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2ba809b870 0x7f2ba809dd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fd0 tx=0x7f2bb0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:13:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:50.915+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2bb8090440 con 0x7f2bc00fd8f0 2023-10-07T23:13:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:13:51.106+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bc002b250 con 0x7f2ba809b870 2023-10-07T23:13:51.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:51 smithi123 ceph-mon[122926]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:51.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:51.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:51.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:51 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:51.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:51 smithi086 ceph-mon[122131]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:51.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:51.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:51.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:51 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:51.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:51 smithi067 ceph-mon[82723]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:51.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:13:51.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:13:51.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:51 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:13:52.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:52 smithi086 ceph-mon[122131]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:52.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:52 smithi067 ceph-mon[82723]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:52.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:52 smithi123 ceph-mon[122926]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:13:53.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:53 smithi086 ceph-mon[122131]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:53.769 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:53 smithi067 ceph-mon[82723]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:53.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:53 smithi123 ceph-mon[122926]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:54.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:54 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/1286162483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]: dispatch 2023-10-07T23:13:54.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:54 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]: dispatch 2023-10-07T23:13:54.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:54 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]': finished 2023-10-07T23:13:54.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:54 smithi086 ceph-mon[122131]: osdmap e69: 11 total, 10 up, 11 in 2023-10-07T23:13:54.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:54 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:13:54.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:54 smithi086 ceph-mon[122131]: from='client.? 172.21.15.86:0/3107195060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:54.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/1286162483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]: dispatch 2023-10-07T23:13:54.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:54 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]: dispatch 2023-10-07T23:13:54.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:54 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]': finished 2023-10-07T23:13:54.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:54 smithi067 ceph-mon[82723]: osdmap e69: 11 total, 10 up, 11 in 2023-10-07T23:13:54.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:54 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:13:54.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.86:0/3107195060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:54.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:54 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/1286162483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]: dispatch 2023-10-07T23:13:54.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:54 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]: dispatch 2023-10-07T23:13:54.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:54 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c315d67a-7a94-4907-9201-c76d2459b5f7"}]': finished 2023-10-07T23:13:54.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:54 smithi123 ceph-mon[122926]: osdmap e69: 11 total, 10 up, 11 in 2023-10-07T23:13:54.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:54 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:13:54.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:54 smithi123 ceph-mon[122926]: from='client.? 172.21.15.86:0/3107195060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:13:55.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:55 smithi086 ceph-mon[122131]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:55.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:55 smithi067 ceph-mon[82723]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:55.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:55 smithi123 ceph-mon[122926]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:57.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:57 smithi086 ceph-mon[122131]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:57.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:57 smithi067 ceph-mon[82723]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:57.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:57 smithi123 ceph-mon[122926]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:59.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:13:59 smithi086 ceph-mon[122131]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:59.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:13:59 smithi067 ceph-mon[82723]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:13:59.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:13:59 smithi123 ceph-mon[122926]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:01.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:01 smithi086 ceph-mon[122131]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:01.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:01 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-07T23:14:01.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:01 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:01.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:01 smithi067 ceph-mon[82723]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:01.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-07T23:14:01.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:01 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:01.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:01 smithi123 ceph-mon[122926]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:01.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:01 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-07T23:14:01.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:01 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:02.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:02 smithi086 ceph-mon[122131]: Deploying daemon osd.10 on smithi086 2023-10-07T23:14:02.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:02 smithi067 ceph-mon[82723]: Deploying daemon osd.10 on smithi086 2023-10-07T23:14:02.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:02 smithi123 ceph-mon[122926]: Deploying daemon osd.10 on smithi086 2023-10-07T23:14:03.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:03 smithi086 ceph-mon[122131]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:03 smithi067 ceph-mon[82723]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:03.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:03 smithi123 ceph-mon[122926]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:04.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:04 smithi086 ceph-mon[122131]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:05.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:04 smithi123 ceph-mon[122926]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:04 smithi067 ceph-mon[82723]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:06 smithi086 ceph-mon[122131]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:06 smithi067 ceph-mon[82723]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:06 smithi123 ceph-mon[122926]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:09.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:08 smithi086 ceph-mon[122131]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:09.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:08 smithi067 ceph-mon[82723]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:09.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:08 smithi123 ceph-mon[122926]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:09.877 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 10 on host 'smithi086' 2023-10-07T23:14:09.877 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.876+0000 7f2bbcff9700 1 -- 172.21.15.86:0/4244663904 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f2bc002b250 con 0x7f2ba809b870 2023-10-07T23:14:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2ba809b870 msgr2=0x7f2ba809dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2ba809b870 0x7f2ba809dd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fd0 tx=0x7f2bb0009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 msgr2=0x7f2bc0115460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc0115460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800bc30 tx=0x7f2bb8009840 comp rx=0 tx=0).stop 2023-10-07T23:14:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 shutdown_connections 2023-10-07T23:14:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc00fd8f0 0x7f2bc0115460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2ba809b870 0x7f2ba809dd20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bc00fbd50 0x7f2bc010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 --2- 172.21.15.86:0/4244663904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bc00fcf60 0x7f2bc010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 >> 172.21.15.86:0/4244663904 conn(0x7f2bc00f74d0 msgr2=0x7f2bc0100190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 shutdown_connections 2023-10-07T23:14:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-07T23:14:09.878+0000 7f2bc63e6700 1 -- 172.21.15.86:0/4244663904 wait complete. 2023-10-07T23:14:10.687 DEBUG:teuthology.orchestra.run.smithi086:osd.10> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-07T23:14:10.689 INFO:tasks.cephadm:Deploying osd.11 on smithi123 with /dev/vg_nvme/lv_1... 2023-10-07T23:14:10.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-07T23:14:10.716 INFO:journalctl@ceph.osd.10.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:14:10.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:10.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-07T23:14:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:10 smithi086 ceph-mon[122131]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-07T23:14:11.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-07T23:14:11.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:10 smithi067 ceph-mon[82723]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-07T23:14:11.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-07T23:14:11.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:10 smithi123 ceph-mon[122926]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-07T23:14:11.278 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:14:12.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:14:11 smithi086 conmon[158442]: 2023-10-07T23:14:11.849+0000 7f82fda4d700 -1 osd.10 0 waiting for initial osdmap 2023-10-07T23:14:12.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:14:11 smithi086 conmon[158442]: 2023-10-07T23:14:11.862+0000 7f82f6844700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:14:12.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:11 smithi086 ceph-mon[122131]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-07T23:14:12.111 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:11 smithi086 ceph-mon[122131]: osdmap e70: 11 total, 10 up, 11 in 2023-10-07T23:14:12.111 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:12.111 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:11 smithi086 ceph-mon[122131]: from='osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:14:12.111 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:11 smithi086 ceph-mon[122131]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:14:12.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:11 smithi123 ceph-mon[122926]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-07T23:14:12.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:11 smithi123 ceph-mon[122926]: osdmap e70: 11 total, 10 up, 11 in 2023-10-07T23:14:12.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:12.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:11 smithi123 ceph-mon[122926]: from='osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:14:12.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:11 smithi123 ceph-mon[122926]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:14:12.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:11 smithi067 ceph-mon[82723]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-07T23:14:12.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:11 smithi067 ceph-mon[82723]: osdmap e70: 11 total, 10 up, 11 in 2023-10-07T23:14:12.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:12.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:11 smithi067 ceph-mon[82723]: from='osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:14:12.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:11 smithi067 ceph-mon[82723]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2023-10-07T23:14:13.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:14:13.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: osdmap e71: 11 total, 10 up, 11 in 2023-10-07T23:14:13.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:13.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] boot 2023-10-07T23:14:13.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: osdmap e72: 11 total, 11 up, 11 in 2023-10-07T23:14:13.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012782 s, 820 MB/s 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 2023-10-07T23:14:13.156 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-10-07T23:14:13.186 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:14:13.186 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: osdmap e71: 11 total, 10 up, 11 in 2023-10-07T23:14:13.186 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.186 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:13.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] boot 2023-10-07T23:14:13.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: osdmap e72: 11 total, 11 up, 11 in 2023-10-07T23:14:13.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: osdmap e71: 11 total, 10 up, 11 in 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 2.8 GiB used, 891 GiB / 894 GiB avail 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: osd.10 [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] boot 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: osdmap e72: 11 total, 11 up, 11 in 2023-10-07T23:14:13.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-07T23:14:13.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2023-10-07T23:14:13.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:14:13.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:14:13.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:13.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:13 smithi086 ceph-mon[122131]: osdmap e73: 11 total, 11 up, 11 in 2023-10-07T23:14:14.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:14.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:14.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:13 smithi123 ceph-mon[122926]: osdmap e73: 11 total, 11 up, 11 in 2023-10-07T23:14:14.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:14:14.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:14.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:14.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:13 smithi067 ceph-mon[82723]: osdmap e73: 11 total, 11 up, 11 in 2023-10-07T23:14:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:14:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:14 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi086 2023-10-07T23:14:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:14 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi086 to 4293M 2023-10-07T23:14:15.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:14 smithi123 ceph-mon[122926]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:15.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:14 smithi123 ceph-mon[122926]: osdmap e74: 11 total, 11 up, 11 in 2023-10-07T23:14:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:14 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi086 2023-10-07T23:14:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:14 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi086 to 4293M 2023-10-07T23:14:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:14 smithi086 ceph-mon[122131]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:14 smithi086 ceph-mon[122131]: osdmap e74: 11 total, 11 up, 11 in 2023-10-07T23:14:15.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:14 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi086 2023-10-07T23:14:15.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:14 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi086 to 4293M 2023-10-07T23:14:15.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:14 smithi067 ceph-mon[82723]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:15.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:14 smithi067 ceph-mon[82723]: osdmap e74: 11 total, 11 up, 11 in 2023-10-07T23:14:15.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.995+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1105780847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fe970 msgr2=0x7fd4b41052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:15.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.995+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1105780847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fe970 0x7fd4b41052a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009d60 tx=0x7fd4ac004d10 comp rx=0 tx=0).stop 2023-10-07T23:14:15.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.995+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1105780847 shutdown_connections 2023-10-07T23:14:15.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.995+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1105780847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4b40fce50 0x7fd4b40fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:15.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1105780847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fe970 0x7fd4b41052a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:15.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1105780847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 0x7fd4b40fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:16.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1105780847 >> 172.21.15.123:0/1105780847 conn(0x7fd4b40f85d0 msgr2=0x7fd4b40fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:16.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1105780847 shutdown_connections 2023-10-07T23:14:16.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1105780847 wait complete. 2023-10-07T23:14:16.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 Processor -- start 2023-10-07T23:14:16.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- start start 2023-10-07T23:14:16.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 0x7fd4b4071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:16.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 0x7fd4b406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:16.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4b40fe970 0x7fd4b40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:16.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4070c40 con 0x7fd4b40fdfc0 2023-10-07T23:14:16.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4070db0 con 0x7fd4b40fe970 2023-10-07T23:14:16.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4bb4cd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4117d50 con 0x7fd4b40fce50 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4b9269700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 0x7fd4b4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4b9269700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 0x7fd4b4071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38014/0 (socket says 172.21.15.123:38014) 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.996+0000 7fd4b9269700 1 -- 172.21.15.123:0/1376321871 learned_addr learned my addr 172.21.15.123:0/1376321871 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b8a68700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 0x7fd4b406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b9269700 1 -- 172.21.15.123:0/1376321871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4b40fe970 msgr2=0x7fd4b40704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b9269700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4b40fe970 0x7fd4b40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b9269700 1 -- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 msgr2=0x7fd4b406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b9269700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 0x7fd4b406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b9269700 1 -- 172.21.15.123:0/1376321871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac009940 con 0x7fd4b40fce50 2023-10-07T23:14:16.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b8a68700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 0x7fd4b406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:14:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4b9269700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 0x7fd4b4071930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b810 tx=0x7fd4b0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000f5f0 con 0x7fd4b40fce50 2023-10-07T23:14:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4b000fc20 con 0x7fd4b40fce50 2023-10-07T23:14:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b4117fa0 con 0x7fd4b40fce50 2023-10-07T23:14:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0010c80 con 0x7fd4b40fce50 2023-10-07T23:14:16.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.997+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b4118400 con 0x7fd4b40fce50 2023-10-07T23:14:16.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.998+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b4108760 con 0x7fd4b40fce50 2023-10-07T23:14:16.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.999+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd4b000fd90 con 0x7fd4b40fce50 2023-10-07T23:14:16.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:15.999+0000 7fd4a9ffb700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd4a009b7f0 0x7fd4a009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:16.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:16.000+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6690+0+0 (secure 0 0 0) 0x7fd4b00c49a0 con 0x7fd4b40fce50 2023-10-07T23:14:16.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:16.000+0000 7fd4b8a68700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd4a009b7f0 0x7fd4a009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:16.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:16.000+0000 7fd4b8a68700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd4a009b7f0 0x7fd4a009dca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006fd0 tx=0x7fd4a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:16.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:16.003+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd4b0090760 con 0x7fd4b40fce50 2023-10-07T23:14:16.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:16.193+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4b4118620 con 0x7fd4a009b7f0 2023-10-07T23:14:17.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:16 smithi123 ceph-mon[122926]: osdmap e75: 11 total, 11 up, 11 in 2023-10-07T23:14:17.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:16 smithi123 ceph-mon[122926]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:17.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:16 smithi123 ceph-mon[122926]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:14:17.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:14:17.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:14:17.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:16 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:16 smithi086 ceph-mon[122131]: osdmap e75: 11 total, 11 up, 11 in 2023-10-07T23:14:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:16 smithi086 ceph-mon[122131]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:16 smithi086 ceph-mon[122131]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:14:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:14:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:14:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:16 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:17.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:16 smithi067 ceph-mon[82723]: osdmap e75: 11 total, 11 up, 11 in 2023-10-07T23:14:17.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:16 smithi067 ceph-mon[82723]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:17.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:16 smithi067 ceph-mon[82723]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:14:17.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-07T23:14:17.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-07T23:14:17.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:16 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:18.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:17 smithi123 ceph-mon[122926]: osdmap e76: 11 total, 11 up, 11 in 2023-10-07T23:14:18.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:17 smithi086 ceph-mon[122131]: osdmap e76: 11 total, 11 up, 11 in 2023-10-07T23:14:18.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:17 smithi067 ceph-mon[82723]: osdmap e76: 11 total, 11 up, 11 in 2023-10-07T23:14:19.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:18 smithi123 ceph-mon[122926]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-10-07T23:14:19.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:18 smithi086 ceph-mon[122131]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-10-07T23:14:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:18 smithi067 ceph-mon[82723]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-10-07T23:14:20.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:19 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/190302886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]: dispatch 2023-10-07T23:14:20.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:19 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]: dispatch 2023-10-07T23:14:20.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:19 smithi123 ceph-mon[122926]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]': finished 2023-10-07T23:14:20.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:19 smithi123 ceph-mon[122926]: osdmap e77: 12 total, 11 up, 12 in 2023-10-07T23:14:20.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:19 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:20.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:19 smithi123 ceph-mon[122926]: from='client.? 172.21.15.123:0/2061997340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:14:20.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:19 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/190302886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]: dispatch 2023-10-07T23:14:20.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:19 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]: dispatch 2023-10-07T23:14:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:19 smithi086 ceph-mon[122131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]': finished 2023-10-07T23:14:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:19 smithi086 ceph-mon[122131]: osdmap e77: 12 total, 11 up, 12 in 2023-10-07T23:14:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:19 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:19 smithi086 ceph-mon[122131]: from='client.? 172.21.15.123:0/2061997340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:14:20.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:19 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/190302886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]: dispatch 2023-10-07T23:14:20.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:19 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]: dispatch 2023-10-07T23:14:20.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:19 smithi067 ceph-mon[82723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6efe153c-3d19-4e94-b8f2-40f78688c379"}]': finished 2023-10-07T23:14:20.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:19 smithi067 ceph-mon[82723]: osdmap e77: 12 total, 11 up, 12 in 2023-10-07T23:14:20.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:19 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:20.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:19 smithi067 ceph-mon[82723]: from='client.? 172.21.15.123:0/2061997340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-07T23:14:21.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:20 smithi123 ceph-mon[122926]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-10-07T23:14:21.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:20 smithi086 ceph-mon[122131]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-10-07T23:14:21.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:20 smithi067 ceph-mon[82723]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-10-07T23:14:23.231 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:22 smithi123 ceph-mon[122926]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-10-07T23:14:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:22 smithi067 ceph-mon[82723]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-10-07T23:14:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:22 smithi086 ceph-mon[122131]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-10-07T23:14:25.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:24 smithi123 ceph-mon[122926]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-07T23:14:25.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:24 smithi086 ceph-mon[122131]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-07T23:14:25.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:24 smithi067 ceph-mon[82723]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-07T23:14:27.159 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:26 smithi123 ceph-mon[122926]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-07T23:14:27.159 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-07T23:14:27.159 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:26 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:27.160 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:26 smithi123 ceph-mon[122926]: Deploying daemon osd.11 on smithi123 2023-10-07T23:14:27.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:26 smithi067 ceph-mon[82723]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-07T23:14:27.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-07T23:14:27.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:26 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:27.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:26 smithi067 ceph-mon[82723]: Deploying daemon osd.11 on smithi123 2023-10-07T23:14:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:26 smithi086 ceph-mon[122131]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-10-07T23:14:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-07T23:14:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:26 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:26 smithi086 ceph-mon[122131]: Deploying daemon osd.11 on smithi123 2023-10-07T23:14:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:28 smithi067 ceph-mon[82723]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:29.307 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:28 smithi123 ceph-mon[122926]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:28 smithi086 ceph-mon[122131]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:30 smithi067 ceph-mon[82723]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:31.341 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:30 smithi123 ceph-mon[122926]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:30 smithi086 ceph-mon[122131]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:32.850 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:32 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:32.850 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:32 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:32.850 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:32 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:32.851 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:32 smithi123 ceph-mon[122926]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:32.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:32 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:32.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:32 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:32.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:32 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:32.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:32 smithi086 ceph-mon[122131]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:33.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:32 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:33.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:32 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:33.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:32 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:33.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:32 smithi067 ceph-mon[82723]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:35.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.068+0000 7fd4a9ffb700 1 -- 172.21.15.123:0/1376321871 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd4b4118620 con 0x7fd4a009b7f0 2023-10-07T23:14:35.069 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 11 on host 'smithi123' 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd4a009b7f0 msgr2=0x7fd4a009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd4a009b7f0 0x7fd4a009dca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006fd0 tx=0x7fd4a4008040 comp rx=0 tx=0).stop 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 msgr2=0x7fd4b4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 0x7fd4b4071930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b810 tx=0x7fd4b0009250 comp rx=0 tx=0).stop 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 shutdown_connections 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4b40fe970 0x7fd4b40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd4a009b7f0 0x7fd4a009dca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4b40fce50 0x7fd4b4071930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 --2- 172.21.15.123:0/1376321871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4b40fdfc0 0x7fd4b406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 >> 172.21.15.123:0/1376321871 conn(0x7fd4b40f85d0 msgr2=0x7fd4b40fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:35.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 shutdown_connections 2023-10-07T23:14:35.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:35.070+0000 7fd4bb4cd700 1 -- 172.21.15.123:0/1376321871 wait complete. 2023-10-07T23:14:35.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:34 smithi123 ceph-mon[122926]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:35.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:34 smithi086 ceph-mon[122131]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:35.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:34 smithi067 ceph-mon[82723]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:35.744 DEBUG:teuthology.orchestra.run.smithi123:osd.11> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-07T23:14:35.747 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-07T23:14:35.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-07T23:14:35.775 INFO:journalctl@ceph.osd.11.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:14:36.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:36 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:36 smithi067 ceph-mon[82723]: from='osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-07T23:14:36.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:36 smithi067 ceph-mon[82723]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-07T23:14:36.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:36 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:36 smithi123 ceph-mon[122926]: from='osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-07T23:14:36.289 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:36 smithi123 ceph-mon[122926]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-07T23:14:36.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:36 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:36.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:36 smithi086 ceph-mon[122131]: from='osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-07T23:14:36.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:36 smithi086 ceph-mon[122131]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-07T23:14:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:14:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:37 smithi123 ceph-mon[122926]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:37 smithi123 ceph-mon[122926]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-07T23:14:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:37 smithi123 ceph-mon[122926]: osdmap e78: 12 total, 11 up, 12 in 2023-10-07T23:14:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:37 smithi123 ceph-mon[122926]: from='osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:14:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:37 smithi123 ceph-mon[122926]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:14:37.391 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:14:37 smithi123 conmon[160185]: 2023-10-07T23:14:37.143+0000 7fd355e5d700 -1 osd.11 0 waiting for initial osdmap 2023-10-07T23:14:37.392 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:14:37 smithi123 conmon[160185]: 2023-10-07T23:14:37.157+0000 7fd350c58700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:14:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:37 smithi086 ceph-mon[122131]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:37 smithi086 ceph-mon[122131]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-07T23:14:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:37 smithi086 ceph-mon[122131]: osdmap e78: 12 total, 11 up, 12 in 2023-10-07T23:14:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:37.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:37 smithi086 ceph-mon[122131]: from='osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:14:37.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:37 smithi086 ceph-mon[122131]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:14:37.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:37 smithi067 ceph-mon[82723]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:37 smithi067 ceph-mon[82723]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-07T23:14:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:37 smithi067 ceph-mon[82723]: osdmap e78: 12 total, 11 up, 12 in 2023-10-07T23:14:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:37 smithi067 ceph-mon[82723]: from='osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:14:37.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:37 smithi067 ceph-mon[82723]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.112+0000 7ff20d9a0700 1 -- 172.21.15.67:0/678833680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 msgr2=0x7ff2080fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.112+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/678833680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 0x7ff2080fd270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80049c0 tx=0x7ff1f8009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.112+0000 7ff20d9a0700 1 -- 172.21.15.67:0/678833680 shutdown_connections 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.112+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/678833680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2080fe980 0x7ff2081052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.112+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/678833680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 0x7ff2080fd270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.112+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/678833680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff2080fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.113+0000 7ff20d9a0700 1 -- 172.21.15.67:0/678833680 >> 172.21.15.67:0/678833680 conn(0x7ff2080f85c0 msgr2=0x7ff2080fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.113+0000 7ff20d9a0700 1 -- 172.21.15.67:0/678833680 shutdown_connections 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.113+0000 7ff20d9a0700 1 -- 172.21.15.67:0/678833680 wait complete. 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.113+0000 7ff20d9a0700 1 Processor -- start 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.113+0000 7ff20d9a0700 1 -- start start 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff20d9a0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 0x7ff208071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff20d9a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff20806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff20d9a0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2080fe980 0x7ff2080704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff20d9a0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff208070c40 con 0x7ff2080fce60 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff20d9a0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff208070db0 con 0x7ff2080fe980 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff20d9a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff208117c70 con 0x7ff2080fdfd0 2023-10-07T23:14:38.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff20806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff20806ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:41084/0 (socket says 172.21.15.67:41084) 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 -- 172.21.15.67:0/2633725741 learned_addr learned my addr 172.21.15.67:0/2633725741 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff206ffd700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 0x7ff208071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2077fe700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2080fe980 0x7ff2080704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 -- 172.21.15.67:0/2633725741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2080fe980 msgr2=0x7ff2080704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2080fe980 0x7ff2080704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 -- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 msgr2=0x7ff208071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 0x7ff208071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.114+0000 7ff2067fc700 1 -- 172.21.15.67:0/2633725741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f8005040 con 0x7ff2080fdfd0 2023-10-07T23:14:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.115+0000 7ff2067fc700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff20806ff90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00ebc0 tx=0x7ff1fc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.115+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc007cc0 con 0x7ff2080fdfd0 2023-10-07T23:14:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.115+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1fc00f040 con 0x7ff2080fdfd0 2023-10-07T23:14:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.115+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff208117ec0 con 0x7ff2080fdfd0 2023-10-07T23:14:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.115+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0094a0 con 0x7ff2080fdfd0 2023-10-07T23:14:38.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.115+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2081183a0 con 0x7ff2080fdfd0 2023-10-07T23:14:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.116+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff208103d10 con 0x7ff2080fdfd0 2023-10-07T23:14:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.118+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7ff1fc00c600 con 0x7ff2080fdfd0 2023-10-07T23:14:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.119+0000 7ff1ef7fe700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff1f009b770 0x7ff1f009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.119+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6843+0+0 (secure 0 0 0) 0x7ff1fc0c4620 con 0x7ff2080fdfd0 2023-10-07T23:14:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.119+0000 7ff206ffd700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff1f009b770 0x7ff1f009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:38.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.120+0000 7ff206ffd700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff1f009b770 0x7ff1f009dc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8004890 tx=0x7ff1f8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:38.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.121+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7ff1fc04da00 con 0x7ff2080fdfd0 2023-10-07T23:14:38.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.327+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff20802b420 con 0x7ff2080fdfd0 2023-10-07T23:14:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.328+0000 7ff1ef7fe700 1 -- 172.21.15.67:0/2633725741 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff1fc019090 con 0x7ff2080fdfd0 2023-10-07T23:14:38.329 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff1f009b770 msgr2=0x7ff1f009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff1f009b770 0x7ff1f009dc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8004890 tx=0x7ff1f8009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 msgr2=0x7ff20806ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff20806ff90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00ebc0 tx=0x7ff1fc00b040 comp rx=0 tx=0).stop 2023-10-07T23:14:38.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 shutdown_connections 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2080fe980 0x7ff2080704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff1f009b770 0x7ff1f009dc20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2080fce60 0x7ff208071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 --2- 172.21.15.67:0/2633725741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2080fdfd0 0x7ff20806ff90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.330+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 >> 172.21.15.67:0/2633725741 conn(0x7ff2080f85c0 msgr2=0x7ff2080fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.331+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 shutdown_connections 2023-10-07T23:14:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:38.331+0000 7ff20d9a0700 1 -- 172.21.15.67:0/2633725741 wait complete. 2023-10-07T23:14:38.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:38 smithi123 ceph-mon[122926]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:14:38.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:38 smithi123 ceph-mon[122926]: osdmap e79: 12 total, 11 up, 12 in 2023-10-07T23:14:38.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:38.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:38 smithi086 ceph-mon[122131]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:14:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:38 smithi086 ceph-mon[122131]: osdmap e79: 12 total, 11 up, 12 in 2023-10-07T23:14:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:38.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:38 smithi067 ceph-mon[82723]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-10-07T23:14:38.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:38 smithi067 ceph-mon[82723]: osdmap e79: 12 total, 11 up, 12 in 2023-10-07T23:14:38.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:38.487 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:38.693 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1696720478,"num_in_osds":12,"osd_in_since":1696720458,"num_remapped_pgs":0} 2023-10-07T23:14:38.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-07T23:14:39.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: purged_snaps scrub starts 2023-10-07T23:14:39.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: purged_snaps scrub ok 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] boot 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: osdmap e80: 12 total, 12 up, 12 in 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2633725741' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:39.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:39 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:14:39.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: purged_snaps scrub starts 2023-10-07T23:14:39.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: purged_snaps scrub ok 2023-10-07T23:14:39.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] boot 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: osdmap e80: 12 total, 12 up, 12 in 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2633725741' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:39.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:39 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: purged_snaps scrub starts 2023-10-07T23:14:39.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: purged_snaps scrub ok 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 3.1 GiB used, 980 GiB / 983 GiB avail 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: osd.11 [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] boot 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: osdmap e80: 12 total, 12 up, 12 in 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2633725741' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:39.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:39 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:40.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:40 smithi086 ceph-mon[122131]: Detected new or changed devices on smithi123 2023-10-07T23:14:40.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:40 smithi086 ceph-mon[122131]: Adjusting osd_memory_target on smithi123 to 4293M 2023-10-07T23:14:40.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:40 smithi086 ceph-mon[122131]: osdmap e81: 12 total, 12 up, 12 in 2023-10-07T23:14:40.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:40 smithi067 ceph-mon[82723]: Detected new or changed devices on smithi123 2023-10-07T23:14:40.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:40 smithi067 ceph-mon[82723]: Adjusting osd_memory_target on smithi123 to 4293M 2023-10-07T23:14:40.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:40 smithi067 ceph-mon[82723]: osdmap e81: 12 total, 12 up, 12 in 2023-10-07T23:14:40.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:40 smithi123 ceph-mon[122926]: Detected new or changed devices on smithi123 2023-10-07T23:14:40.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:40 smithi123 ceph-mon[122926]: Adjusting osd_memory_target on smithi123 to 4293M 2023-10-07T23:14:40.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:40 smithi123 ceph-mon[122926]: osdmap e81: 12 total, 12 up, 12 in 2023-10-07T23:14:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.881+0000 7f886f377700 1 -- 172.21.15.67:0/1790556071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88681037e0 msgr2=0x7f88680fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.881+0000 7f886f377700 1 --2- 172.21.15.67:0/1790556071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88681037e0 0x7f88680fce50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88580049f0 tx=0x7f8858009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 -- 172.21.15.67:0/1790556071 shutdown_connections 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 --2- 172.21.15.67:0/1790556071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88680fd530 0x7f8868109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 --2- 172.21.15.67:0/1790556071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8868102e40 0x7f8868103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 --2- 172.21.15.67:0/1790556071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88681037e0 0x7f88680fce50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 -- 172.21.15.67:0/1790556071 >> 172.21.15.67:0/1790556071 conn(0x7f88680f89f0 msgr2=0x7f88680fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 -- 172.21.15.67:0/1790556071 shutdown_connections 2023-10-07T23:14:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 -- 172.21.15.67:0/1790556071 wait complete. 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.882+0000 7f886f377700 1 Processor -- start 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 -- start start 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88680fd530 0x7f8868115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 0x7f8868116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88681037e0 0x7f886810fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681103b0 con 0x7f8868102e40 2023-10-07T23:14:40.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868110520 con 0x7f88680fd530 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886f377700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681106e0 con 0x7f88681037e0 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886c912700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 0x7f8868116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886c912700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 0x7f8868116220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35114/0 (socket says 172.21.15.67:35114) 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886d914700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88681037e0 0x7f886810fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886c912700 1 -- 172.21.15.67:0/912717382 learned_addr learned my addr 172.21.15.67:0/912717382 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.883+0000 7f886d113700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88680fd530 0x7f8868115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886c912700 1 -- 172.21.15.67:0/912717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88681037e0 msgr2=0x7f886810fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886c912700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88681037e0 0x7f886810fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886c912700 1 -- 172.21.15.67:0/912717382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88680fd530 msgr2=0x7f8868115ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886c912700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88680fd530 0x7f8868115ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886c912700 1 -- 172.21.15.67:0/912717382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858005040 con 0x7f8868102e40 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886d914700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88681037e0 0x7f886810fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886c912700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 0x7f8868116220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f885800bfc0 tx=0x7f8858009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800e040 con 0x7f8868102e40 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88580096f0 con 0x7f8868102e40 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858013d60 con 0x7f8868102e40 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8868110960 con 0x7f8868102e40 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.884+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886811ccd0 con 0x7f8868102e40 2023-10-07T23:14:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.886+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8868063f20 con 0x7f8868102e40 2023-10-07T23:14:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.888+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f88580134b0 con 0x7f8868102e40 2023-10-07T23:14:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.890+0000 7f885dffb700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f885409b770 0x7f885409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.890+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f88580c65c0 con 0x7f8868102e40 2023-10-07T23:14:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.890+0000 7f886d113700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f885409b770 0x7f885409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.891+0000 7f886d113700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f885409b770 0x7f885409dc20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8864004f60 tx=0x7f886400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:40.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:40.892+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f88580921d0 con 0x7f8868102e40 2023-10-07T23:14:41.096 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.097+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f886810ccb0 con 0x7f8868102e40 2023-10-07T23:14:41.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.098+0000 7f885dffb700 1 -- 172.21.15.67:0/912717382 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18101 (secure 0 0 0) 0x7f88580957f0 con 0x7f8868102e40 2023-10-07T23:14:41.098 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:41.098 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:14:39.148128+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f885409b770 msgr2=0x7f885409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f885409b770 0x7f885409dc20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8864004f60 tx=0x7f886400b040 comp rx=0 tx=0).stop 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 msgr2=0x7f8868116220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 0x7f8868116220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f885800bfc0 tx=0x7f8858009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 shutdown_connections 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88680fd530 0x7f8868115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f885409b770 0x7f885409dc20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88681037e0 0x7f886810fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 --2- 172.21.15.67:0/912717382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8868102e40 0x7f8868116220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:41.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 >> 172.21.15.67:0/912717382 conn(0x7f88680f89f0 msgr2=0x7f886806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 shutdown_connections 2023-10-07T23:14:41.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:41.100+0000 7f886f377700 1 -- 172.21.15.67:0/912717382 wait complete. 2023-10-07T23:14:41.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:41 smithi086 ceph-mon[122131]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:41.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:41 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/912717382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:14:41.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:41 smithi067 ceph-mon[82723]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:41.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:41 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/912717382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:14:41.539 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:14:41.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-07T23:14:41.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:41 smithi123 ceph-mon[122926]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:41.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:41 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/912717382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:14:42.309 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:14:43.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:43 smithi086 ceph-mon[122131]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:43.450 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:43 smithi067 ceph-mon[82723]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:43.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:43 smithi123 ceph-mon[122926]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 -- 172.21.15.67:0/3123672645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e406fe70 msgr2=0x7f06e4070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 --2- 172.21.15.67:0/3123672645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e406fe70 0x7f06e4070240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f06e0004b10 tx=0x7f06e0009dd0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 -- 172.21.15.67:0/3123672645 shutdown_connections 2023-10-07T23:14:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 --2- 172.21.15.67:0/3123672645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 0x7f06e410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 --2- 172.21.15.67:0/3123672645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e406fe70 0x7f06e4070240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 --2- 172.21.15.67:0/3123672645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e4070780 0x7f06e41093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 -- 172.21.15.67:0/3123672645 >> 172.21.15.67:0/3123672645 conn(0x7f06e4073e90 msgr2=0x7f06e4074290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:43.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 -- 172.21.15.67:0/3123672645 shutdown_connections 2023-10-07T23:14:43.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.708+0000 7f06ebe02700 1 -- 172.21.15.67:0/3123672645 wait complete. 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.709+0000 7f06ebe02700 1 Processor -- start 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.709+0000 7f06ebe02700 1 -- start start 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ebe02700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e406fe70 0x7f06e4115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ebe02700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 0x7f06e4116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ebe02700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 0x7f06e410fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ebe02700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41103d0 con 0x7f06e4070780 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ebe02700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e4110540 con 0x7f06e41098e0 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ebe02700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e4110700 con 0x7f06e406fe70 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 0x7f06e4116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 0x7f06e4116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46868/0 (socket says 172.21.15.67:46868) 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 -- 172.21.15.67:0/574132970 learned_addr learned my addr 172.21.15.67:0/574132970 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e9b9e700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e406fe70 0x7f06e4115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ea39f700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 0x7f06e410fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 -- 172.21.15.67:0/574132970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e406fe70 msgr2=0x7f06e4115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e406fe70 0x7f06e4115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 -- 172.21.15.67:0/574132970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 msgr2=0x7f06e410fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 0x7f06e410fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06e939d700 1 -- 172.21.15.67:0/574132970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e0004650 con 0x7f06e4070780 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.710+0000 7f06ea39f700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 0x7f06e410fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:14:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.711+0000 7f06e939d700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 0x7f06e4116240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06d400ebc0 tx=0x7f06d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.711+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4007cc0 con 0x7f06e4070780 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.711+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e4110980 con 0x7f06e4070780 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.711+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d400f040 con 0x7f06e4070780 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.711+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400c8c0 con 0x7f06e4070780 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.711+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e411ccf0 con 0x7f06e4070780 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.712+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e4063f20 con 0x7f06e4070780 2023-10-07T23:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.714+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f06d400ccf0 con 0x7f06e4070780 2023-10-07T23:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.715+0000 7f06da7fc700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f06d009b770 0x7f06d009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.716+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f06d40c4950 con 0x7f06e4070780 2023-10-07T23:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.716+0000 7f06e9b9e700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f06d009b770 0x7f06d009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.716+0000 7f06e9b9e700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f06d009b770 0x7f06d009dc20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009fd0 tx=0x7f06e0016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.718+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f06d4090560 con 0x7f06e4070780 2023-10-07T23:14:43.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.924+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06e411ce80 con 0x7f06e4070780 2023-10-07T23:14:43.925 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.924+0000 7f06da7fc700 1 -- 172.21.15.67:0/574132970 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06d4023020 con 0x7f06e4070780 2023-10-07T23:14:43.925 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:14:43.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.926+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f06d009b770 msgr2=0x7f06d009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:43.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.926+0000 7f06ebe02700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f06d009b770 0x7f06d009dc20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009fd0 tx=0x7f06e0016040 comp rx=0 tx=0).stop 2023-10-07T23:14:43.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 msgr2=0x7f06e4116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:43.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 0x7f06e4116240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06d400ebc0 tx=0x7f06d400b040 comp rx=0 tx=0).stop 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 shutdown_connections 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e41098e0 0x7f06e410fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f06d009b770 0x7f06d009dc20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06e406fe70 0x7f06e4115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 --2- 172.21.15.67:0/574132970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06e4070780 0x7f06e4116240 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 >> 172.21.15.67:0/574132970 conn(0x7f06e4073e90 msgr2=0x7f06e41033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 shutdown_connections 2023-10-07T23:14:43.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:43.927+0000 7f06ebe02700 1 -- 172.21.15.67:0/574132970 wait complete. 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.a on smithi067 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.d on smithi067 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.g on smithi067 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.j on smithi067 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.b on smithi086 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.e on smithi086 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.h on smithi086 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.k on smithi086 2023-10-07T23:14:44.377 INFO:tasks.cephadm:Adding mds.c on smithi123 2023-10-07T23:14:44.378 INFO:tasks.cephadm:Adding mds.f on smithi123 2023-10-07T23:14:44.378 INFO:tasks.cephadm:Adding mds.i on smithi123 2023-10-07T23:14:44.378 INFO:tasks.cephadm:Adding mds.l on smithi123 2023-10-07T23:14:44.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l' 2023-10-07T23:14:44.969 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-07T23:14:45.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:44 smithi123 ceph-mon[122926]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:45.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:44 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/574132970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:14:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:44 smithi086 ceph-mon[122131]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:44 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/574132970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:14:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:44 smithi067 ceph-mon[82723]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:44 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/574132970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:14:46.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 -- 172.21.15.123:0/1115756168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff80ff060 msgr2=0x7f1ff80ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/1115756168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff80ff060 0x7f1ff80ff470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80049c0 tx=0x7f1fe8009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 -- 172.21.15.123:0/1115756168 shutdown_connections 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/1115756168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff8100c00 0x7f1ff81074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/1115756168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff80ff060 0x7f1ff80ff470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/1115756168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 0x7f1ff81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 -- 172.21.15.123:0/1115756168 >> 172.21.15.123:0/1115756168 conn(0x7f1ff80fa7c0 msgr2=0x7f1ff80fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 -- 172.21.15.123:0/1115756168 shutdown_connections 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.549+0000 7f1ffca2d700 1 -- 172.21.15.123:0/1115756168 wait complete. 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 Processor -- start 2023-10-07T23:14:46.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 -- start start 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff80ff060 0x7f1ff810fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 0x7f1ff81103e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 0x7f1ff8117470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8117a80 con 0x7f1ff80ff060 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8117bf0 con 0x7f1ff8100270 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8117db0 con 0x7f1ff8100c00 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 0x7f1ff8117470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 0x7f1ff8117470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38144/0 (socket says 172.21.15.123:38144) 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 -- 172.21.15.123:0/3271687840 learned_addr learned my addr 172.21.15.123:0/3271687840 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff659c700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff80ff060 0x7f1ff810fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff5d9b700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 0x7f1ff81103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 -- 172.21.15.123:0/3271687840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 msgr2=0x7f1ff81103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 0x7f1ff81103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 -- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff80ff060 msgr2=0x7f1ff810fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff80ff060 0x7f1ff810fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 -- 172.21.15.123:0/3271687840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8005040 con 0x7f1ff8100c00 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff659c700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff80ff060 0x7f1ff810fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff5d9b700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 0x7f1ff81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ff6d9d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 0x7f1ff8117470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000bc30 tx=0x7f1ff0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00092e0 con 0x7f1ff8100c00 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff000cc70 con 0x7f1ff8100c00 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff8118090 con 0x7f1ff8100c00 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.550+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0015d50 con 0x7f1ff8100c00 2023-10-07T23:14:46.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.551+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8118530 con 0x7f1ff8100c00 2023-10-07T23:14:46.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.551+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff810a950 con 0x7f1ff8100c00 2023-10-07T23:14:46.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.552+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f1ff0012070 con 0x7f1ff8100c00 2023-10-07T23:14:46.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.553+0000 7f1fe6ffd700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1fe009b770 0x7f1fe009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:46.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.553+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f1ff00c4840 con 0x7f1ff8100c00 2023-10-07T23:14:46.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.553+0000 7f1ff659c700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1fe009b770 0x7f1fe009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:46.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.554+0000 7f1ff659c700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1fe009b770 0x7f1fe009dc20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8004890 tx=0x7f1fe8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.557+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f1ff00904f0 con 0x7f1ff8100c00 2023-10-07T23:14:46.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.747+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ff805e180 con 0x7f1fe009b770 2023-10-07T23:14:46.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.755+0000 7f1fe6ffd700 1 -- 172.21.15.123:0/3271687840 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f1ff805e180 con 0x7f1fe009b770 2023-10-07T23:14:46.756 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mds.all update... 2023-10-07T23:14:46.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1fe009b770 msgr2=0x7f1fe009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:46.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1fe009b770 0x7f1fe009dc20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8004890 tx=0x7f1fe8009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:46.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 msgr2=0x7f1ff8117470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:46.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 0x7f1ff8117470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000bc30 tx=0x7f1ff0009840 comp rx=0 tx=0).stop 2023-10-07T23:14:46.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 shutdown_connections 2023-10-07T23:14:46.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1fe009b770 0x7f1fe009dc20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ff8100c00 0x7f1ff8117470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ff80ff060 0x7f1ff810fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 --2- 172.21.15.123:0/3271687840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff8100270 0x7f1ff81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:46.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.757+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 >> 172.21.15.123:0/3271687840 conn(0x7f1ff80fa7c0 msgr2=0x7f1ff80fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:46.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.758+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 shutdown_connections 2023-10-07T23:14:46.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-07T23:14:46.758+0000 7f1ffca2d700 1 -- 172.21.15.123:0/3271687840 wait complete. 2023-10-07T23:14:47.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:46 smithi123 ceph-mon[122926]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:47.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:47.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:46 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:46 smithi067 ceph-mon[82723]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:46 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:47.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:46 smithi086 ceph-mon[122131]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:47.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:47.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:46 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:14:47.504 DEBUG:teuthology.orchestra.run.smithi067:mds.a> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-07T23:14:47.509 DEBUG:teuthology.orchestra.run.smithi067:mds.d> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-07T23:14:47.554 DEBUG:teuthology.orchestra.run.smithi067:mds.g> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-07T23:14:47.579 INFO:journalctl@ceph.mds.a.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:14:47.582 DEBUG:teuthology.orchestra.run.smithi067:mds.j> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-07T23:14:47.617 INFO:journalctl@ceph.mds.d.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:14:47.620 DEBUG:teuthology.orchestra.run.smithi086:mds.b> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-07T23:14:47.623 DEBUG:teuthology.orchestra.run.smithi086:mds.e> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-07T23:14:47.643 INFO:journalctl@ceph.mds.g.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:14:47.651 INFO:journalctl@ceph.mds.b.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:14:47.652 DEBUG:teuthology.orchestra.run.smithi086:mds.h> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-07T23:14:47.675 INFO:journalctl@ceph.mds.e.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:14:47.675 DEBUG:teuthology.orchestra.run.smithi086:mds.k> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-07T23:14:47.681 INFO:journalctl@ceph.mds.j.smithi067.stdout:-- Logs begin at Sat 2023-10-07 22:51:32 UTC. -- 2023-10-07T23:14:47.697 INFO:journalctl@ceph.mds.h.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:14:47.698 DEBUG:teuthology.orchestra.run.smithi123:mds.c> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-07T23:14:47.700 DEBUG:teuthology.orchestra.run.smithi123:mds.f> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-07T23:14:47.721 INFO:journalctl@ceph.mds.k.smithi086.stdout:-- Logs begin at Sat 2023-10-07 22:47:19 UTC. -- 2023-10-07T23:14:47.727 INFO:journalctl@ceph.mds.c.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:14:47.728 DEBUG:teuthology.orchestra.run.smithi123:mds.i> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-07T23:14:47.751 INFO:journalctl@ceph.mds.f.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:14:47.752 DEBUG:teuthology.orchestra.run.smithi123:mds.l> sudo journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-07T23:14:47.775 INFO:journalctl@ceph.mds.i.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:14:47.776 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-07T23:14:47.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:14:47.800 INFO:journalctl@ceph.mds.l.smithi123.stdout:-- Logs begin at Sat 2023-10-07 22:47:29 UTC. -- 2023-10-07T23:14:47.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.974+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3562779557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4071bc0 msgr2=0x7f10c4070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.974+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3562779557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4071bc0 0x7f10c4070030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10b80049c0 tx=0x7f10b8009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3562779557 shutdown_connections 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3562779557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4104ba0 0x7f10c4107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3562779557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4071bc0 0x7f10c4070030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3562779557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c4070570 0x7f10c40709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3562779557 >> 172.21.15.67:0/3562779557 conn(0x7f10c40f8420 msgr2=0x7f10c40fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3562779557 shutdown_connections 2023-10-07T23:14:47.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3562779557 wait complete. 2023-10-07T23:14:47.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 Processor -- start 2023-10-07T23:14:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.975+0000 7f10c9d9d700 1 -- start start 2023-10-07T23:14:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c9d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 0x7f10c410fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c9d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4071bc0 0x7f10c4110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c9d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c4104ba0 0x7f10c41172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c9d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4117900 con 0x7f10c4070570 2023-10-07T23:14:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c9d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4117a70 con 0x7f10c4071bc0 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c9d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4117c30 con 0x7f10c4104ba0 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 0x7f10c410fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 0x7f10c410fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46876/0 (socket says 172.21.15.67:46876) 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 -- 172.21.15.67:0/3206494541 learned_addr learned my addr 172.21.15.67:0/3206494541 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c959c700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c4104ba0 0x7f10c41172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c3fff700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4071bc0 0x7f10c4110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 -- 172.21.15.67:0/3206494541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c4104ba0 msgr2=0x7f10c41172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c4104ba0 0x7f10c41172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 -- 172.21.15.67:0/3206494541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4071bc0 msgr2=0x7f10c4110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4071bc0 0x7f10c4110260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c8d9b700 1 -- 172.21.15.67:0/3206494541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8005040 con 0x7f10c4070570 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.976+0000 7f10c3fff700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4071bc0 0x7f10c4110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.977+0000 7f10c8d9b700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 0x7f10c410fd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009180 tx=0x7f10b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.977+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8020070 con 0x7f10c4070570 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.977+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b8009520 con 0x7f10c4070570 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.977+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b800e070 con 0x7f10c4070570 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.977+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c4117eb0 con 0x7f10c4070570 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.977+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c4118200 con 0x7f10c4070570 2023-10-07T23:14:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.978+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c410a750 con 0x7f10c4070570 2023-10-07T23:14:47.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.980+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f10b8009690 con 0x7f10c4070570 2023-10-07T23:14:47.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.981+0000 7f10c17fa700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f10b409b7f0 0x7f10b409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:47.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.981+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f10b80c6430 con 0x7f10c4070570 2023-10-07T23:14:47.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.982+0000 7f10c3fff700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f10b409b7f0 0x7f10b409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:47.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.982+0000 7f10c3fff700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f10b409b7f0 0x7f10b409dca0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10b0004fd0 tx=0x7f10b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:47.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:47.983+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f10b80920c0 con 0x7f10c4070570 2023-10-07T23:14:48.030 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:48.031 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:47 smithi123 ceph-mon[122926]: Deploying daemon mds.l on smithi123 2023-10-07T23:14:48.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.190+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10c4118450 con 0x7f10c4070570 2023-10-07T23:14:48.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.191+0000 7f10c17fa700 1 -- 172.21.15.67:0/3206494541 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18101 (secure 0 0 0) 0x7f10b80956e0 con 0x7f10c4070570 2023-10-07T23:14:48.192 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:48.192 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:14:39.148128+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:14:48.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f10b409b7f0 msgr2=0x7f10b409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f10b409b7f0 0x7f10b409dca0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10b0004fd0 tx=0x7f10b000b040 comp rx=0 tx=0).stop 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 msgr2=0x7f10c410fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 0x7f10c410fd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009180 tx=0x7f10b8009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 shutdown_connections 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f10b409b7f0 0x7f10b409dca0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c4104ba0 0x7f10c41172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c4070570 0x7f10c410fd20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 --2- 172.21.15.67:0/3206494541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c4071bc0 0x7f10c4110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 >> 172.21.15.67:0/3206494541 conn(0x7f10c40f8420 msgr2=0x7f10c4106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 shutdown_connections 2023-10-07T23:14:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.194+0000 7f10c9d9d700 1 -- 172.21.15.67:0/3206494541 wait complete. 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:48.211 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:47 smithi067 ceph-mon[82723]: Deploying daemon mds.l on smithi123 2023-10-07T23:14:48.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:14:48.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:14:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 -- 172.21.15.67:0/634427491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fdf00 msgr2=0x7f77a40fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 --2- 172.21.15.67:0/634427491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fdf00 0x7f77a40fe350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7794004900 tx=0x7f7794009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 -- 172.21.15.67:0/634427491 shutdown_connections 2023-10-07T23:14:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 --2- 172.21.15.67:0/634427491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40fe890 0x7f77a4105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 --2- 172.21.15.67:0/634427491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a40fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 --2- 172.21.15.67:0/634427491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fdf00 0x7f77a40fe350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.404+0000 7f77aa19d700 1 -- 172.21.15.67:0/634427491 >> 172.21.15.67:0/634427491 conn(0x7f77a40f8450 msgr2=0x7f77a40fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.405+0000 7f77aa19d700 1 -- 172.21.15.67:0/634427491 shutdown_connections 2023-10-07T23:14:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.405+0000 7f77aa19d700 1 -- 172.21.15.67:0/634427491 wait complete. 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.405+0000 7f77aa19d700 1 Processor -- start 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.405+0000 7f77aa19d700 1 -- start start 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77aa19d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77aa19d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40fdf00 0x7f77a410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77aa19d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fe890 0x7f77a4114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77aa19d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4115540 con 0x7f77a40fccf0 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77aa19d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41156b0 con 0x7f77a40fe890 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77aa19d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4115870 con 0x7f77a40fdf00 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46896/0 (socket says 172.21.15.67:46896) 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 -- 172.21.15.67:0/3033023687 learned_addr learned my addr 172.21.15.67:0/3033023687 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a899a700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40fdf00 0x7f77a410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a999c700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fe890 0x7f77a4114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 -- 172.21.15.67:0/3033023687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40fdf00 msgr2=0x7f77a410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40fdf00 0x7f77a410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 -- 172.21.15.67:0/3033023687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fe890 msgr2=0x7f77a4114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fe890 0x7f77a4114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.406+0000 7f77a919b700 1 -- 172.21.15.67:0/3033023687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0005040 con 0x7f77a40fccf0 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.407+0000 7f77a919b700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a410d960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009730 tx=0x7f77a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.407+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0007be0 con 0x7f77a40fccf0 2023-10-07T23:14:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.407+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a000f040 con 0x7f77a40fccf0 2023-10-07T23:14:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.407+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7794005040 con 0x7f77a40fccf0 2023-10-07T23:14:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.407+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0013c30 con 0x7f77a40fccf0 2023-10-07T23:14:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.407+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a4115f20 con 0x7f77a40fccf0 2023-10-07T23:14:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.408+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4103b60 con 0x7f77a40fccf0 2023-10-07T23:14:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.410+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f77a000e840 con 0x7f77a40fccf0 2023-10-07T23:14:48.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.411+0000 7f7799ffb700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f779009b7f0 0x7f779009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.411+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f77a00c49a0 con 0x7f77a40fccf0 2023-10-07T23:14:48.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.411+0000 7f77a899a700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f779009b7f0 0x7f779009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.412+0000 7f77a899a700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f779009b7f0 0x7f779009dca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7794004f70 tx=0x7f7794009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:48.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.415+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f77a00905b0 con 0x7f77a40fccf0 2023-10-07T23:14:48.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:14:48.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-10-07T23:14:48.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:14:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:47 smithi086 ceph-mon[122131]: Deploying daemon mds.l on smithi123 2023-10-07T23:14:48.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.615+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77a402b510 con 0x7f77a40fccf0 2023-10-07T23:14:48.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.616+0000 7f7799ffb700 1 -- 172.21.15.67:0/3033023687 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77a0093bd0 con 0x7f77a40fccf0 2023-10-07T23:14:48.617 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f779009b7f0 msgr2=0x7f779009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f779009b7f0 0x7f779009dca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7794004f70 tx=0x7f7794009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 msgr2=0x7f77a410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a410d960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009730 tx=0x7f77a0009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 shutdown_connections 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77a40fe890 0x7f77a4114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f779009b7f0 0x7f779009dca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a40fccf0 0x7f77a410d960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 --2- 172.21.15.67:0/3033023687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a40fdf00 0x7f77a410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 >> 172.21.15.67:0/3033023687 conn(0x7f77a40f8450 msgr2=0x7f77a40fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:48.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 shutdown_connections 2023-10-07T23:14:48.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.618+0000 7f77aa19d700 1 -- 172.21.15.67:0/3033023687 wait complete. 2023-10-07T23:14:48.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:14:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 -- 172.21.15.67:0/1129672798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fccf0 msgr2=0x7f92500fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 --2- 172.21.15.67:0/1129672798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fccf0 0x7f92500fd100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f924c0049c0 tx=0x7f924c009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 -- 172.21.15.67:0/1129672798 shutdown_connections 2023-10-07T23:14:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 --2- 172.21.15.67:0/1129672798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 0x7f9250105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 --2- 172.21.15.67:0/1129672798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fccf0 0x7f92500fd100 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 --2- 172.21.15.67:0/1129672798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fdf00 0x7f92500fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 -- 172.21.15.67:0/1129672798 >> 172.21.15.67:0/1129672798 conn(0x7f92500f8410 msgr2=0x7f92500fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 -- 172.21.15.67:0/1129672798 shutdown_connections 2023-10-07T23:14:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.824+0000 7f925719d700 1 -- 172.21.15.67:0/1129672798 wait complete. 2023-10-07T23:14:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.825+0000 7f925719d700 1 Processor -- start 2023-10-07T23:14:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.825+0000 7f925719d700 1 -- start start 2023-10-07T23:14:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.825+0000 7f925719d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 0x7f925010fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.825+0000 7f925719d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fdf00 0x7f92501100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.825+0000 7f925719d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 0x7f9250117150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.825+0000 7f925719d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9250117760 con 0x7f92500fccf0 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925719d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92501178d0 con 0x7f92500fe890 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925719d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9250117a90 con 0x7f92500fdf00 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 0x7f925010fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 0x7f925010fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46912/0 (socket says 172.21.15.67:46912) 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 -- 172.21.15.67:0/3666413417 learned_addr learned my addr 172.21.15.67:0/3666413417 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925599a700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fdf00 0x7f92501100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925699c700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 0x7f9250117150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 -- 172.21.15.67:0/3666413417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fdf00 msgr2=0x7f92501100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fdf00 0x7f92501100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 -- 172.21.15.67:0/3666413417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 msgr2=0x7f9250117150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 0x7f9250117150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 -- 172.21.15.67:0/3666413417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c005040 con 0x7f92500fccf0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925699c700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 0x7f9250117150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925599a700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fdf00 0x7f92501100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.826+0000 7f925619b700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 0x7f925010fb80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f924c0137e0 tx=0x7f924c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.827+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00e050 con 0x7f92500fccf0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.827+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9250117d10 con 0x7f92500fccf0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.827+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f924c0097b0 con 0x7f92500fccf0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.827+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c01be40 con 0x7f92500fccf0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.827+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9250118170 con 0x7f92500fccf0 2023-10-07T23:14:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.828+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9250108550 con 0x7f92500fccf0 2023-10-07T23:14:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.830+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f924c01b4f0 con 0x7f92500fccf0 2023-10-07T23:14:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.831+0000 7f9246ffd700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f923c09b770 0x7f923c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.831+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f924c0c61d0 con 0x7f92500fccf0 2023-10-07T23:14:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.831+0000 7f925599a700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f923c09b770 0x7f923c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.832+0000 7f925599a700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f923c09b770 0x7f923c09dc20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9240009b80 tx=0x7f924000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:48.835+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f924c091e80 con 0x7f92500fccf0 2023-10-07T23:14:49.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.085+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9250118390 con 0x7f92500fccf0 2023-10-07T23:14:49.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.085+0000 7f9246ffd700 1 -- 172.21.15.67:0/3666413417 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f924c01f210 con 0x7f92500fccf0 2023-10-07T23:14:49.086 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:49.086 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-07T23:14:49.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f923c09b770 msgr2=0x7f923c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f923c09b770 0x7f923c09dc20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9240009b80 tx=0x7f924000b040 comp rx=0 tx=0).stop 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 msgr2=0x7f925010fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 0x7f925010fb80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f924c0137e0 tx=0x7f924c009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 shutdown_connections 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92500fe890 0x7f9250117150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f923c09b770 0x7f923c09dc20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92500fccf0 0x7f925010fb80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 --2- 172.21.15.67:0/3666413417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92500fdf00 0x7f92501100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 >> 172.21.15.67:0/3666413417 conn(0x7f92500f8410 msgr2=0x7f9250103760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 shutdown_connections 2023-10-07T23:14:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:49.088+0000 7f925719d700 1 -- 172.21.15.67:0/3666413417 wait complete. 2023-10-07T23:14:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 1 2023-10-07T23:14:49.235 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:48 smithi123 ceph-mon[122926]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:49.235 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:48 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3206494541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:14:49.235 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:48 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3033023687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:14:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:48 smithi067 ceph-mon[82723]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:48 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3206494541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:14:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:48 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3033023687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:14:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:48 smithi086 ceph-mon[122131]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:48 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3206494541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:14:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:48 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3033023687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:14:49.563 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:14:49 smithi123 systemd[1]: Starting Ceph mds.l for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:14:49.890 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mds[166646]: -- 172.21.15.123:0/1945180193 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d8c7309c0 con 0x560d8c706c00 2023-10-07T23:14:49.890 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:14:49 smithi123 conmon[166622]: starting mds.l at 2023-10-07T23:14:49.891 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:14:49 smithi123 bash[166582]: baede428ce2633f9cb30151acdbbf862c150b22e682d7358468524c5dac3eeb7 2023-10-07T23:14:49.891 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:14:49 smithi123 systemd[1]: Started Ceph mds.l for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:14:50.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3666413417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:50.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] up:boot 2023-10-07T23:14:50.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: fsmap 1 up:standby 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:50.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:49 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:50.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3666413417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] up:boot 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: fsmap 1 up:standby 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:50.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:49 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:50.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3666413417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:50.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] up:boot 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: fsmap 1 up:standby 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:49 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:51.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:14:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:50 smithi067 ceph-mon[82723]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:50 smithi067 ceph-mon[82723]: Deploying daemon mds.k on smithi086 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/3850336431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 msgr2=0x7f5c380fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/3850336431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c380fd0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c28004900 tx=0x7f5c28009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/3850336431 shutdown_connections 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/3850336431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c380fd0f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/3850336431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c380fe880 0x7f5c38105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/3850336431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c380fdef0 0x7f5c380fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/3850336431 >> 172.21.15.67:0/3850336431 conn(0x7f5c380f8460 msgr2=0x7f5c380fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/3850336431 shutdown_connections 2023-10-07T23:14:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.308+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/3850336431 wait complete. 2023-10-07T23:14:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 Processor -- start 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 -- start start 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c3810fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c380fdef0 0x7f5c381101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c380fe880 0x7f5c381171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38117830 con 0x7f5c380fdef0 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381179a0 con 0x7f5c380fcce0 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c3cb9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38117b60 con 0x7f5c380fe880 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c377fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c3810fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c377fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c3810fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:60668/0 (socket says 172.21.15.67:60668) 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c377fe700 1 -- 172.21.15.67:0/2659042938 learned_addr learned my addr 172.21.15.67:0/2659042938 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.309+0000 7f5c36ffd700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c380fdef0 0x7f5c381101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c37fff700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c380fe880 0x7f5c381171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c377fe700 1 -- 172.21.15.67:0/2659042938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c380fe880 msgr2=0x7f5c381171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c377fe700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c380fe880 0x7f5c381171f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c377fe700 1 -- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c380fdef0 msgr2=0x7f5c381101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c377fe700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c380fdef0 0x7f5c381101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c377fe700 1 -- 172.21.15.67:0/2659042938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28005040 con 0x7f5c380fcce0 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c377fe700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c3810fc60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c28013980 tx=0x7f5c28009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800e040 con 0x7f5c380fcce0 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2801c430 con 0x7f5c380fcce0 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28012630 con 0x7f5c380fcce0 2023-10-07T23:14:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c38117de0 con 0x7f5c380fcce0 2023-10-07T23:14:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.310+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c38118190 con 0x7f5c380fcce0 2023-10-07T23:14:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.311+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c38108540 con 0x7f5c380fcce0 2023-10-07T23:14:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.313+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f5c2801c5a0 con 0x7f5c380fcce0 2023-10-07T23:14:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.314+0000 7f5c1ffff700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5c2009b770 0x7f5c2009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.314+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f5c280c65f0 con 0x7f5c380fcce0 2023-10-07T23:14:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.315+0000 7f5c36ffd700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5c2009b770 0x7f5c2009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.315+0000 7f5c36ffd700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5c2009b770 0x7f5c2009dc20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009b80 tx=0x7f5c2c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.316+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f5c28092200 con 0x7f5c380fcce0 2023-10-07T23:14:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:50 smithi123 ceph-mon[122926]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:50 smithi123 ceph-mon[122926]: Deploying daemon mds.k on smithi086 2023-10-07T23:14:51.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:50 smithi086 ceph-mon[122131]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:51.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:50 smithi086 ceph-mon[122131]: Deploying daemon mds.k on smithi086 2023-10-07T23:14:51.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.563+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c38118450 con 0x7f5c380fcce0 2023-10-07T23:14:51.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.564+0000 7f5c1ffff700 1 -- 172.21.15.67:0/2659042938 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f5c28095820 con 0x7f5c380fcce0 2023-10-07T23:14:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:51.565 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-07T23:14:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5c2009b770 msgr2=0x7f5c2009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5c2009b770 0x7f5c2009dc20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009b80 tx=0x7f5c2c00b040 comp rx=0 tx=0).stop 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 msgr2=0x7f5c3810fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c3810fc60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c28013980 tx=0x7f5c28009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 shutdown_connections 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380fcce0 0x7f5c3810fc60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5c2009b770 0x7f5c2009dc20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c380fe880 0x7f5c381171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 --2- 172.21.15.67:0/2659042938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c380fdef0 0x7f5c381101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 >> 172.21.15.67:0/2659042938 conn(0x7f5c380f8460 msgr2=0x7f5c38103780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 shutdown_connections 2023-10-07T23:14:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:51.567+0000 7f5c3cb9d700 1 -- 172.21.15.67:0/2659042938 wait complete. 2023-10-07T23:14:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2023-10-07T23:14:51.773 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:14:51 smithi086 systemd[1]: Starting Ceph mds.k for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:14:52.234 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:51 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2659042938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:52.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:51 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2659042938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:52.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:51 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2659042938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:52.698 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mds[164728]: -- 172.21.15.86:0/1151709427 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8326589c0 con 0x55a83262ec00 2023-10-07T23:14:52.698 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:14:52 smithi086 conmon[164704]: starting mds.k at 2023-10-07T23:14:52.698 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:14:52 smithi086 bash[164663]: e9decf3230a82e0cfe39e21f0c05b7534589dc60c4087c4928a0670a37042953 2023-10-07T23:14:52.698 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:14:52 smithi086 systemd[1]: Started Ceph mds.k for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:14:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:52 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:53.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:52 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:52 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:53.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:14:53.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 -- 172.21.15.67:0/262315775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fcf00 msgr2=0x7fd1e00fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:53.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/262315775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fcf00 0x7fd1e00fd310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0049c0 tx=0x7fd1dc009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:53.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 -- 172.21.15.67:0/262315775 shutdown_connections 2023-10-07T23:14:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/262315775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e00feb60 0x7fd1e0105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/262315775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fcf00 0x7fd1e00fd310 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/262315775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e00fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 -- 172.21.15.67:0/262315775 >> 172.21.15.67:0/262315775 conn(0x7fd1e00f8660 msgr2=0x7fd1e00faa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 -- 172.21.15.67:0/262315775 shutdown_connections 2023-10-07T23:14:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.789+0000 7fd1e79ff700 1 -- 172.21.15.67:0/262315775 wait complete. 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 Processor -- start 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 -- start start 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fcf00 0x7fd1e0102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e0103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e00feb60 0x7fd1e00ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0100260 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e01003d0 con 0x7fd1e00feb60 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e79ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0100590 con 0x7fd1e00fcf00 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e4f9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e0103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e4f9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e0103490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55410/0 (socket says 172.21.15.67:55410) 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e4f9a700 1 -- 172.21.15.67:0/326595561 learned_addr learned my addr 172.21.15.67:0/326595561 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e5f9c700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e00feb60 0x7fd1e00ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.791+0000 7fd1e579b700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fcf00 0x7fd1e0102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e4f9a700 1 -- 172.21.15.67:0/326595561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fcf00 msgr2=0x7fd1e0102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e4f9a700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fcf00 0x7fd1e0102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e4f9a700 1 -- 172.21.15.67:0/326595561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e00feb60 msgr2=0x7fd1e00ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e4f9a700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e00feb60 0x7fd1e00ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e4f9a700 1 -- 172.21.15.67:0/326595561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d80048f0 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e579b700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fcf00 0x7fd1e0102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:14:53.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e4f9a700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e0103490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000ebf0 tx=0x7fd1d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:53.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0007cc0 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc005040 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d000f040 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0009460 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.792+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e011c670 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.793+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e0108730 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.797+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd1d000c680 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.800+0000 7fd1d5ffb700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd1cc09b770 0x7fd1cc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:53.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.800+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fd1d00c4710 con 0x7fd1e00fe1b0 2023-10-07T23:14:53.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.800+0000 7fd1e579b700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd1cc09b770 0x7fd1cc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:53.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.801+0000 7fd1e579b700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd1cc09b770 0x7fd1cc09dc20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc007f50 tx=0x7fd1dc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:53.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:53.803+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd1d00903c0 con 0x7fd1e00fe1b0 2023-10-07T23:14:54.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.065+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1e011c890 con 0x7fd1e00fe1b0 2023-10-07T23:14:54.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.066+0000 7fd1d5ffb700 1 -- 172.21.15.67:0/326595561 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7fd1d0019800 con 0x7fd1e00fe1b0 2023-10-07T23:14:54.066 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:54.067 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd1cc09b770 msgr2=0x7fd1cc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd1cc09b770 0x7fd1cc09dc20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc007f50 tx=0x7fd1dc009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 msgr2=0x7fd1e0103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e0103490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000ebf0 tx=0x7fd1d000b040 comp rx=0 tx=0).stop 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 shutdown_connections 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1e00feb60 0x7fd1e00ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd1cc09b770 0x7fd1cc09dc20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1e00fcf00 0x7fd1e0102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 --2- 172.21.15.67:0/326595561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e00fe1b0 0x7fd1e0103490 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.068+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 >> 172.21.15.67:0/326595561 conn(0x7fd1e00f8660 msgr2=0x7fd1e00fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.069+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 shutdown_connections 2023-10-07T23:14:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:54.069+0000 7fd1e79ff700 1 -- 172.21.15.67:0/326595561 wait complete. 2023-10-07T23:14:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2023-10-07T23:14:54.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:53 smithi067 ceph-mon[82723]: Deploying daemon mds.j on smithi067 2023-10-07T23:14:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:53 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6832/1133310774,v1:172.21.15.86:6833/1133310774] up:boot 2023-10-07T23:14:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:53 smithi067 ceph-mon[82723]: fsmap 2 up:standby 2023-10-07T23:14:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:53 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-07T23:14:54.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:53 smithi123 ceph-mon[122926]: Deploying daemon mds.j on smithi067 2023-10-07T23:14:54.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:53 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6832/1133310774,v1:172.21.15.86:6833/1133310774] up:boot 2023-10-07T23:14:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:53 smithi123 ceph-mon[122926]: fsmap 2 up:standby 2023-10-07T23:14:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:53 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-07T23:14:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:53 smithi086 ceph-mon[122131]: Deploying daemon mds.j on smithi067 2023-10-07T23:14:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:53 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6832/1133310774,v1:172.21.15.86:6833/1133310774] up:boot 2023-10-07T23:14:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:53 smithi086 ceph-mon[122131]: fsmap 2 up:standby 2023-10-07T23:14:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:53 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-07T23:14:54.990 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:14:54 smithi067 systemd[1]: Starting Ceph mds.j for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:14:55.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:54 smithi067 ceph-mon[82723]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:55.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/326595561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:55.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:54 smithi123 ceph-mon[122926]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:54 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/326595561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:55.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:54 smithi086 ceph-mon[122131]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:55.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:54 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/326595561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:55.754 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:14:55 smithi067 bash[137746]: c7b6b3ba964106c79132cab6059a3fbebc17bee1ef677cd875369e6736862212 2023-10-07T23:14:55.754 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:14:55 smithi067 ceph-mds[137811]: -- 172.21.15.67:0/545102378 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd4859a9c0 con 0x55cd48570c00 2023-10-07T23:14:55.754 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:14:55 smithi067 conmon[137786]: starting mds.j at 2023-10-07T23:14:55.755 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:14:55 smithi067 systemd[1]: Started Ceph mds.j for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:14:56.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 -- 172.21.15.67:0/676663624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c1000f0 msgr2=0x7f101c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 --2- 172.21.15.67:0/676663624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c1000f0 0x7f101c100560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10100049c0 tx=0x7f1010009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 -- 172.21.15.67:0/676663624 shutdown_connections 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 --2- 172.21.15.67:0/676663624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c100aa0 0x7f101c1073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 --2- 172.21.15.67:0/676663624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c0fef00 0x7f101c0ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 --2- 172.21.15.67:0/676663624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c1000f0 0x7f101c100560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 -- 172.21.15.67:0/676663624 >> 172.21.15.67:0/676663624 conn(0x7f101c0fa790 msgr2=0x7f101c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 -- 172.21.15.67:0/676663624 shutdown_connections 2023-10-07T23:14:56.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.295+0000 7f101b59e700 1 -- 172.21.15.67:0/676663624 wait complete. 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 Processor -- start 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 -- start start 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 0x7f101c10fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1000f0 0x7f101c1102d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c100aa0 0x7f101c117360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c117970 con 0x7f101c0fef00 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c117ae0 con 0x7f101c100aa0 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.296+0000 7f101b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c117ca0 con 0x7f101c1000f0 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f101a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 0x7f101c10fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f1019d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1000f0 0x7f101c1102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f101a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 0x7f101c10fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55458/0 (socket says 172.21.15.67:55458) 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f101a59c700 1 -- 172.21.15.67:0/635867736 learned_addr learned my addr 172.21.15.67:0/635867736 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f101ad9d700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c100aa0 0x7f101c117360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f1019d9b700 1 -- 172.21.15.67:0/635867736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c100aa0 msgr2=0x7f101c117360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f1019d9b700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c100aa0 0x7f101c117360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f1019d9b700 1 -- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 msgr2=0x7f101c10fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f1019d9b700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 0x7f101c10fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f1019d9b700 1 -- 172.21.15.67:0/635867736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140048f0 con 0x7f101c1000f0 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.297+0000 7f101a59c700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 0x7f101c10fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.298+0000 7f1019d9b700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1000f0 0x7f101c1102d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10100049c0 tx=0x7f1010009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.298+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000e070 con 0x7f101c1000f0 2023-10-07T23:14:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.298+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1010009870 con 0x7f101c1000f0 2023-10-07T23:14:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.298+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1010005040 con 0x7f101c1000f0 2023-10-07T23:14:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.298+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100123c0 con 0x7f101c1000f0 2023-10-07T23:14:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.298+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c1181a0 con 0x7f101c1000f0 2023-10-07T23:14:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.299+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c106230 con 0x7f101c1000f0 2023-10-07T23:14:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.301+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f10100093a0 con 0x7f101c1000f0 2023-10-07T23:14:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.302+0000 7f100affd700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f100409b770 0x7f100409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.302+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f10100c5d40 con 0x7f101c1000f0 2023-10-07T23:14:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.303+0000 7f101a59c700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f100409b770 0x7f100409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.303+0000 7f101a59c700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f100409b770 0x7f100409dc20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f100c009b80 tx=0x7f100c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:56.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.306+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f1010091950 con 0x7f101c1000f0 2023-10-07T23:14:56.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.575+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f101c02b220 con 0x7f101c1000f0 2023-10-07T23:14:56.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.576+0000 7f100affd700 1 -- 172.21.15.67:0/635867736 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7f1010094f70 con 0x7f101c1000f0 2023-10-07T23:14:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:56.577 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f100409b770 msgr2=0x7f100409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f100409b770 0x7f100409dc20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f100c009b80 tx=0x7f100c00b040 comp rx=0 tx=0).stop 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1000f0 msgr2=0x7f101c1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1000f0 0x7f101c1102d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10100049c0 tx=0x7f1010009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 shutdown_connections 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c100aa0 0x7f101c117360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f100409b770 0x7f100409dc20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f101c0fef00 0x7f101c10fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 --2- 172.21.15.67:0/635867736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1000f0 0x7f101c1102d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:56.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 >> 172.21.15.67:0/635867736 conn(0x7f101c0fa790 msgr2=0x7f101c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:56.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.578+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 shutdown_connections 2023-10-07T23:14:56.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:56.579+0000 7f101b59e700 1 -- 172.21.15.67:0/635867736 wait complete. 2023-10-07T23:14:56.580 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 4 2023-10-07T23:14:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:56.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: Deploying daemon mds.d on smithi067 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6834/959405817,v1:172.21.15.67:6835/959405817] up:boot 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: fsmap 3 up:standby 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-07T23:14:56.892 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:56 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/635867736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:56.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:56.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:56.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:56.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: Deploying daemon mds.d on smithi067 2023-10-07T23:14:56.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:56.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6834/959405817,v1:172.21.15.67:6835/959405817] up:boot 2023-10-07T23:14:56.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: fsmap 3 up:standby 2023-10-07T23:14:56.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-07T23:14:56.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:56 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/635867736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:56.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: Deploying daemon mds.d on smithi067 2023-10-07T23:14:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6834/959405817,v1:172.21.15.67:6835/959405817] up:boot 2023-10-07T23:14:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: fsmap 3 up:standby 2023-10-07T23:14:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-07T23:14:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:56 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/635867736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:14:58.255 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:14:57 smithi067 systemd[1]: Starting Ceph mds.d for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:14:58.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:14:58.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 -- 172.21.15.67:0/656866756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00fefc0 msgr2=0x7f5cc00ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:58.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 --2- 172.21.15.67:0/656866756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00fefc0 0x7f5cc00ff3d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4004900 tx=0x7f5cb4009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:58.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 -- 172.21.15.67:0/656866756 shutdown_connections 2023-10-07T23:14:58.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 --2- 172.21.15.67:0/656866756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc0100b60 0x7f5cc0107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:58.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 --2- 172.21.15.67:0/656866756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00fefc0 0x7f5cc00ff3d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:58.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 --2- 172.21.15.67:0/656866756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc01001d0 0x7f5cc0100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:58.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 -- 172.21.15.67:0/656866756 >> 172.21.15.67:0/656866756 conn(0x7f5cc00fa630 msgr2=0x7f5cc00fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:58.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 -- 172.21.15.67:0/656866756 shutdown_connections 2023-10-07T23:14:58.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.805+0000 7f5cc599d700 1 -- 172.21.15.67:0/656866756 wait complete. 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.806+0000 7f5cc599d700 1 Processor -- start 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.806+0000 7f5cc599d700 1 -- start start 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.806+0000 7f5cc599d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc00fefc0 0x7f5cc010fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.806+0000 7f5cc599d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 0x7f5cc01101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cc599d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0100b60 0x7f5cc0117240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cc599d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0117890 con 0x7f5cc0100b60 2023-10-07T23:14:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cc599d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0117a00 con 0x7f5cc01001d0 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cc599d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0117bc0 con 0x7f5cc00fefc0 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 0x7f5cc01101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 0x7f5cc01101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:45266/0 (socket says 172.21.15.67:45266) 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 -- 172.21.15.67:0/586164996 learned_addr learned my addr 172.21.15.67:0/586164996 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cc519c700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0100b60 0x7f5cc0117240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cc499b700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc00fefc0 0x7f5cc010fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 -- 172.21.15.67:0/586164996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc00fefc0 msgr2=0x7f5cc010fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc00fefc0 0x7f5cc010fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 -- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0100b60 msgr2=0x7f5cc0117240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0100b60 0x7f5cc0117240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 -- 172.21.15.67:0/586164996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8009940 con 0x7f5cc01001d0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.807+0000 7f5cbffff700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 0x7f5cc01101c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cac002ad0 tx=0x7f5cac0046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.808+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac007cc0 con 0x7f5cc01001d0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.808+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb4005040 con 0x7f5cc01001d0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.808+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cac00f040 con 0x7f5cc01001d0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.808+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00d900 con 0x7f5cc01001d0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.808+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc0118230 con 0x7f5cc01001d0 2023-10-07T23:14:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.809+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc010a820 con 0x7f5cc01001d0 2023-10-07T23:14:58.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.811+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f5cac00dd30 con 0x7f5cc01001d0 2023-10-07T23:14:58.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.812+0000 7f5cbd7fa700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5cb009b770 0x7f5cb009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:14:58.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.812+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f5cac0c4c50 con 0x7f5cc01001d0 2023-10-07T23:14:58.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.812+0000 7f5cc499b700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5cb009b770 0x7f5cb009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:14:58.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.813+0000 7f5cc499b700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5cb009b770 0x7f5cb009dc20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009040 tx=0x7f5cb4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:14:58.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:58.818+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f5cac0908f0 con 0x7f5cc01001d0 2023-10-07T23:14:58.969 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:14:58 smithi067 bash[138349]: 94fc296308b7fe43b61ccf4c5059f0726f90e074b6b0ce512d074533f08b8a31 2023-10-07T23:14:58.969 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mds[138413]: -- 172.21.15.67:0/1365619444 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556bea02e9c0 con 0x556bea004800 2023-10-07T23:14:58.969 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:14:58 smithi067 conmon[138389]: starting mds.d at 2023-10-07T23:14:58.969 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:14:58 smithi067 systemd[1]: Started Ceph mds.d for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:14:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.068+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc0118420 con 0x7f5cc01001d0 2023-10-07T23:14:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.069+0000 7f5cbd7fa700 1 -- 172.21.15.67:0/586164996 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3525 (secure 0 0 0) 0x7f5cac017de0 con 0x7f5cc01001d0 2023-10-07T23:14:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:14:59.070 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2023-10-07T23:14:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5cb009b770 msgr2=0x7f5cb009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5cb009b770 0x7f5cb009dc20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009040 tx=0x7f5cb4009e30 comp rx=0 tx=0).stop 2023-10-07T23:14:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 msgr2=0x7f5cc01101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:14:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 0x7f5cc01101c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cac002ad0 tx=0x7f5cac0046b0 comp rx=0 tx=0).stop 2023-10-07T23:14:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 shutdown_connections 2023-10-07T23:14:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f5cb009b770 0x7f5cb009dc20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0100b60 0x7f5cc0117240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc00fefc0 0x7f5cc010fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 --2- 172.21.15.67:0/586164996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cc01001d0 0x7f5cc01101c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.071+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 >> 172.21.15.67:0/586164996 conn(0x7f5cc00fa630 msgr2=0x7f5cc00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.072+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 shutdown_connections 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:14:59.072+0000 7f5cc599d700 1 -- 172.21.15.67:0/586164996 wait complete. 2023-10-07T23:14:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-10-07T23:14:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:58 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:59.338 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:58 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:14:59.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:14:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:14:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:14:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:14:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:58 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:00.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:59 smithi086 ceph-mon[122131]: Deploying daemon mds.e on smithi086 2023-10-07T23:15:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:59 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] up:boot 2023-10-07T23:15:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:59 smithi086 ceph-mon[122131]: fsmap 4 up:standby 2023-10-07T23:15:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:59 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-07T23:15:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:14:59 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/586164996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:00.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:59 smithi067 ceph-mon[82723]: Deploying daemon mds.e on smithi086 2023-10-07T23:15:00.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:59 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] up:boot 2023-10-07T23:15:00.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:59 smithi067 ceph-mon[82723]: fsmap 4 up:standby 2023-10-07T23:15:00.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:59 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-07T23:15:00.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:14:59 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/586164996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:00.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:59 smithi123 ceph-mon[122926]: Deploying daemon mds.e on smithi086 2023-10-07T23:15:00.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:59 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] up:boot 2023-10-07T23:15:00.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:59 smithi123 ceph-mon[122926]: fsmap 4 up:standby 2023-10-07T23:15:00.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:59 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-07T23:15:00.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:14:59 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/586164996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:00.672 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:15:00 smithi086 systemd[1]: Starting Ceph mds.e for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:01.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:01.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:00 smithi067 ceph-mon[82723]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:01.288 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:00 smithi086 ceph-mon[122131]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 -- 172.21.15.67:0/3777135705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fcd00 msgr2=0x7fa6dc0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 --2- 172.21.15.67:0/3777135705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc0fd110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80049c0 tx=0x7fa6d8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 -- 172.21.15.67:0/3777135705 shutdown_connections 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 --2- 172.21.15.67:0/3777135705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fe8a0 0x7fa6dc105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 --2- 172.21.15.67:0/3777135705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc0fd110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 --2- 172.21.15.67:0/3777135705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6dc0fdf10 0x7fa6dc0fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 -- 172.21.15.67:0/3777135705 >> 172.21.15.67:0/3777135705 conn(0x7fa6dc0f8420 msgr2=0x7fa6dc0fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 -- 172.21.15.67:0/3777135705 shutdown_connections 2023-10-07T23:15:01.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.292+0000 7fa6e219d700 1 -- 172.21.15.67:0/3777135705 wait complete. 2023-10-07T23:15:01.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.293+0000 7fa6e219d700 1 Processor -- start 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.293+0000 7fa6e219d700 1 -- start start 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.293+0000 7fa6e219d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e219d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fdf10 0x7fa6dc10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e219d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6dc0fe8a0 0x7fa6dc114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e219d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc115600 con 0x7fa6dc0fe8a0 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e219d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc115770 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e219d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc115930 con 0x7fa6dc0fdf10 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc10da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:45284/0 (socket says 172.21.15.67:45284) 2023-10-07T23:15:01.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 -- 172.21.15.67:0/180545423 learned_addr learned my addr 172.21.15.67:0/180545423 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e199c700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6dc0fe8a0 0x7fa6dc114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e099a700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fdf10 0x7fa6dc10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 -- 172.21.15.67:0/180545423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fdf10 msgr2=0x7fa6dc10df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fdf10 0x7fa6dc10df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 -- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6dc0fe8a0 msgr2=0x7fa6dc114ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6dc0fe8a0 0x7fa6dc114ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 -- 172.21.15.67:0/180545423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc005040 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6e119b700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc10da20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009150 tx=0x7fa6d8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.294+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8020070 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.295+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8005040 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.295+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6d80041a0 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.295+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800e070 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.295+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc115f00 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.296+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc104280 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.297+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fa6d80095a0 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.299+0000 7fa6d1ffb700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa6c809b770 0x7fa6c809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.299+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fa6d80c61e0 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.299+0000 7fa6e099a700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa6c809b770 0x7fa6c809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:01.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.299+0000 7fa6e099a700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa6c809b770 0x7fa6c809dc20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009fd0 tx=0x7fa6cc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:01.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.300+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fa6d8091df0 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:00 smithi123 ceph-mon[122926]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.551+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6dc108560 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.551+0000 7fa6d1ffb700 1 -- 172.21.15.67:0/180545423 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3525 (secure 0 0 0) 0x7fa6d8095410 con 0x7fa6dc0fcd00 2023-10-07T23:15:01.552 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:01.552 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa6c809b770 msgr2=0x7fa6c809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa6c809b770 0x7fa6c809dc20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009fd0 tx=0x7fa6cc009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 msgr2=0x7fa6dc10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc10da20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009150 tx=0x7fa6d8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 shutdown_connections 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc0fcd00 0x7fa6dc10da20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa6c809b770 0x7fa6c809dc20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6dc0fe8a0 0x7fa6dc114ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 --2- 172.21.15.67:0/180545423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6dc0fdf10 0x7fa6dc10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 >> 172.21.15.67:0/180545423 conn(0x7fa6dc0f8420 msgr2=0x7fa6dc101400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 shutdown_connections 2023-10-07T23:15:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:01.554+0000 7fa6e219d700 1 -- 172.21.15.67:0/180545423 wait complete. 2023-10-07T23:15:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-10-07T23:15:01.697 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:15:01 smithi086 ceph-mds[165291]: -- 172.21.15.86:0/2302764656 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e5e91589c0 con 0x55e5e912f000 2023-10-07T23:15:01.697 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:15:01 smithi086 conmon[165267]: starting mds.e at 2023-10-07T23:15:01.697 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:15:01 smithi086 bash[165227]: 3af8eaca59f3962465b5f03d96f0439478e9dca76e3dc06b3e74c8f058c37b7c 2023-10-07T23:15:01.697 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:15:01 smithi086 systemd[1]: Started Ceph mds.e for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:02.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/180545423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6834/3322109112,v1:172.21.15.86:6835/3322109112] up:boot 2023-10-07T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: fsmap 5 up:standby 2023-10-07T23:15:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:02 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-07T23:15:02.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/180545423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6834/3322109112,v1:172.21.15.86:6835/3322109112] up:boot 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: fsmap 5 up:standby 2023-10-07T23:15:02.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:02 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-07T23:15:02.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.890 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/180545423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6834/3322109112,v1:172.21.15.86:6835/3322109112] up:boot 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: fsmap 5 up:standby 2023-10-07T23:15:02.891 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:02 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-07T23:15:03.391 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:15:03 smithi123 systemd[1]: Starting Ceph mds.f for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:03.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:03.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:03 smithi086 ceph-mon[122131]: Deploying daemon mds.f on smithi123 2023-10-07T23:15:03.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:03 smithi086 ceph-mon[122131]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:03.705 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:03 smithi123 ceph-mon[122926]: Deploying daemon mds.f on smithi123 2023-10-07T23:15:03.705 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:03 smithi123 ceph-mon[122926]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:03 smithi067 ceph-mon[82723]: Deploying daemon mds.f on smithi123 2023-10-07T23:15:03.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:03 smithi067 ceph-mon[82723]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 -- 172.21.15.67:0/629051545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fdef0 msgr2=0x7f43b40fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 --2- 172.21.15.67:0/629051545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fdef0 0x7f43b40fe340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f43a8004900 tx=0x7f43a8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 -- 172.21.15.67:0/629051545 shutdown_connections 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 --2- 172.21.15.67:0/629051545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b40fcce0 0x7f43b40fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 --2- 172.21.15.67:0/629051545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fe880 0x7f43b4105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 --2- 172.21.15.67:0/629051545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fdef0 0x7f43b40fe340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 -- 172.21.15.67:0/629051545 >> 172.21.15.67:0/629051545 conn(0x7f43b40f8460 msgr2=0x7f43b40fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:03.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 -- 172.21.15.67:0/629051545 shutdown_connections 2023-10-07T23:15:03.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.774+0000 7f43b3fff700 1 -- 172.21.15.67:0/629051545 wait complete. 2023-10-07T23:15:03.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 Processor -- start 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 -- start start 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b40fcce0 0x7f43b410fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 0x7f43b41100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 0x7f43b4117130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4117740 con 0x7f43b40fdef0 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41178b0 con 0x7f43b40fcce0 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.775+0000 7f43b3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4117a70 con 0x7f43b40fe880 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 0x7f43b41100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 0x7f43b4117130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 0x7f43b4117130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:38608/0 (socket says 172.21.15.67:38608) 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b40fcce0 0x7f43b410fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b37fe700 1 -- 172.21.15.67:0/1337674930 learned_addr learned my addr 172.21.15.67:0/1337674930 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 0x7f43b41100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48386/0 (socket says 172.21.15.67:48386) 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 -- 172.21.15.67:0/1337674930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 msgr2=0x7f43b4117130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 0x7f43b4117130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 -- 172.21.15.67:0/1337674930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b40fcce0 msgr2=0x7f43b410fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b40fcce0 0x7f43b410fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 -- 172.21.15.67:0/1337674930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8005040 con 0x7f43b40fdef0 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b37fe700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 0x7f43b4117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f43b27fc700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 0x7f43b41100a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43a80047d0 tx=0x7f43a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8020070 con 0x7f43b40fdef0 2023-10-07T23:15:03.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43a801c430 con 0x7f43b40fdef0 2023-10-07T23:15:03.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.776+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800e040 con 0x7f43b40fdef0 2023-10-07T23:15:03.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.777+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b4117cf0 con 0x7f43b40fdef0 2023-10-07T23:15:03.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.777+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b41180f0 con 0x7f43b40fdef0 2023-10-07T23:15:03.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.778+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b4108550 con 0x7f43b40fdef0 2023-10-07T23:15:03.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.780+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f43a801c5a0 con 0x7f43b40fdef0 2023-10-07T23:15:03.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.781+0000 7f439b7fe700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f439c09b7f0 0x7f439c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:03.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.781+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f43a80c5fc0 con 0x7f43b40fdef0 2023-10-07T23:15:03.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.781+0000 7f43b2ffd700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f439c09b7f0 0x7f439c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:03.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.782+0000 7f43b2ffd700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f439c09b7f0 0x7f439c09dca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009b80 tx=0x7f43a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:03.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:03.783+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f43a8091cf0 con 0x7f43b40fdef0 2023-10-07T23:15:04.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.036+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43b41182e0 con 0x7f43b40fdef0 2023-10-07T23:15:04.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.037+0000 7f439b7fe700 1 -- 172.21.15.67:0/1337674930 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f43a8095310 con 0x7f43b40fdef0 2023-10-07T23:15:04.038 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:04.038 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-10-07T23:15:04.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.039+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f439c09b7f0 msgr2=0x7f439c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:04.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.039+0000 7f43b3fff700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f439c09b7f0 0x7f439c09dca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009b80 tx=0x7f43a400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:04.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.039+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 msgr2=0x7f43b41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:04.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.039+0000 7f43b3fff700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 0x7f43b41100a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43a80047d0 tx=0x7f43a8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 shutdown_connections 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b40fcce0 0x7f43b410fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f439c09b7f0 0x7f439c09dca0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b40fe880 0x7f43b4117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 --2- 172.21.15.67:0/1337674930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43b40fdef0 0x7f43b41100a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 >> 172.21.15.67:0/1337674930 conn(0x7f43b40f8460 msgr2=0x7f43b40fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 shutdown_connections 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:04.040+0000 7f43b3fff700 1 -- 172.21.15.67:0/1337674930 wait complete. 2023-10-07T23:15:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 6 2023-10-07T23:15:04.140 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:15:03 smithi123 ceph-mds[167208]: -- 172.21.15.123:0/2878529440 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b867adc9c0 con 0x55b867ab2800 2023-10-07T23:15:04.141 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:15:03 smithi123 conmon[167184]: starting mds.f at 2023-10-07T23:15:04.141 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:15:03 smithi123 bash[167145]: 26a37b7ad3c866ae7c7e679d2e1fa8ae19522bc9c2dd9811bfcd15811a6c24f1 2023-10-07T23:15:04.141 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:15:03 smithi123 systemd[1]: Started Ceph mds.f for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: Deploying daemon mds.c on smithi123 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1337674930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6834/4209275840,v1:172.21.15.123:6835/4209275840] up:boot 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: fsmap 6 up:standby 2023-10-07T23:15:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:04 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-07T23:15:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: Deploying daemon mds.c on smithi123 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1337674930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:05.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6834/4209275840,v1:172.21.15.123:6835/4209275840] up:boot 2023-10-07T23:15:05.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: fsmap 6 up:standby 2023-10-07T23:15:05.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:04 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-07T23:15:05.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:05.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: Deploying daemon mds.c on smithi123 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1337674930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6834/4209275840,v1:172.21.15.123:6835/4209275840] up:boot 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: fsmap 6 up:standby 2023-10-07T23:15:05.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:04 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-07T23:15:05.808 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:15:05 smithi123 systemd[1]: Starting Ceph mds.c for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:06.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 -- 172.21.15.67:0/3812741523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fe880 msgr2=0x7f7018105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 --2- 172.21.15.67:0/3812741523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fe880 0x7f7018105140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7010009d60 tx=0x7f7010004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 -- 172.21.15.67:0/3812741523 shutdown_connections 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 --2- 172.21.15.67:0/3812741523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fe880 0x7f7018105140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 --2- 172.21.15.67:0/3812741523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fcce0 0x7f70180fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 --2- 172.21.15.67:0/3812741523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70180fdef0 0x7f70180fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 -- 172.21.15.67:0/3812741523 >> 172.21.15.67:0/3812741523 conn(0x7f70180f8460 msgr2=0x7f70180fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 -- 172.21.15.67:0/3812741523 shutdown_connections 2023-10-07T23:15:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.258+0000 7f7017fff700 1 -- 172.21.15.67:0/3812741523 wait complete. 2023-10-07T23:15:06.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 Processor -- start 2023-10-07T23:15:06.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 -- start start 2023-10-07T23:15:06.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fcce0 0x7f701810d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:06.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70180fdef0 0x7f701810dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:06.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 0x7f7018114d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018115360 con 0x7f70180fcce0 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70181154d0 con 0x7f70180fdef0 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7017fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018115690 con 0x7f70180fe880 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f70177fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 0x7f7018114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f7016ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fcce0 0x7f701810d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f70177fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 0x7f7018114d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:38622/0 (socket says 172.21.15.67:38622) 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f70177fe700 1 -- 172.21.15.67:0/4102499936 learned_addr learned my addr 172.21.15.67:0/4102499936 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.259+0000 7f700ffff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70180fdef0 0x7f701810dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f70177fe700 1 -- 172.21.15.67:0/4102499936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70180fdef0 msgr2=0x7f701810dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f70177fe700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70180fdef0 0x7f701810dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f70177fe700 1 -- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fcce0 msgr2=0x7f701810d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f70177fe700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fcce0 0x7f701810d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f70177fe700 1 -- 172.21.15.67:0/4102499936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000005040 con 0x7f70180fe880 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f70177fe700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 0x7f7018114d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7010004b70 tx=0x7f7010004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701000e030 con 0x7f70180fe880 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7010009940 con 0x7f70180fe880 2023-10-07T23:15:06.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7010009520 con 0x7f70180fe880 2023-10-07T23:15:06.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701001c690 con 0x7f70180fe880 2023-10-07T23:15:06.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.260+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7018115c60 con 0x7f70180fe880 2023-10-07T23:15:06.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.261+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7018104000 con 0x7f70180fe880 2023-10-07T23:15:06.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.263+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f701001cac0 con 0x7f70180fe880 2023-10-07T23:15:06.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.265+0000 7f700f7fe700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f700409b770 0x7f700409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:06.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.265+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f70100c7180 con 0x7f70180fe880 2023-10-07T23:15:06.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.265+0000 7f7016ffd700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f700409b770 0x7f700409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.265+0000 7f7016ffd700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f700409b770 0x7f700409dc20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70180fdd50 tx=0x7f700800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:06.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.266+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7010092d90 con 0x7f70180fe880 2023-10-07T23:15:06.416 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mds[167768]: -- 172.21.15.123:0/2567686406 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2441689c0 con 0x55b24413ec00 2023-10-07T23:15:06.417 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:15:06 smithi123 conmon[167744]: starting mds.c at 2023-10-07T23:15:06.417 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:15:06 smithi123 bash[167704]: 27399760f4fb346faa5c6acc9b55280fd04f10c3cb5f93225cdaea53ecfd8c88 2023-10-07T23:15:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.517+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f701802b2a0 con 0x7f70180fe880 2023-10-07T23:15:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.518+0000 7f700f7fe700 1 -- 172.21.15.67:0/4102499936 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7f7010020090 con 0x7f70180fe880 2023-10-07T23:15:06.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:06.519 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-10-07T23:15:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.520+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f700409b770 msgr2=0x7f700409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.520+0000 7f7017fff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f700409b770 0x7f700409dc20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70180fdd50 tx=0x7f700800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.520+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 msgr2=0x7f7018114d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.520+0000 7f7017fff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 0x7f7018114d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7010004b70 tx=0x7f7010004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 shutdown_connections 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f700409b770 0x7f700409dc20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70180fe880 0x7f7018114d50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70180fcce0 0x7f701810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 --2- 172.21.15.67:0/4102499936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70180fdef0 0x7f701810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 >> 172.21.15.67:0/4102499936 conn(0x7f70180f8460 msgr2=0x7f7018101180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 shutdown_connections 2023-10-07T23:15:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:06.521+0000 7f7017fff700 1 -- 172.21.15.67:0/4102499936 wait complete. 2023-10-07T23:15:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 7 2023-10-07T23:15:06.890 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:15:06 smithi123 systemd[1]: Started Ceph mds.c for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:07.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:07.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:07.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:07.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:07.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:06 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4102499936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:07.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:06 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4102499936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:07.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:06 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4102499936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:08.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:07 smithi067 ceph-mon[82723]: Deploying daemon mds.h on smithi086 2023-10-07T23:15:08.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:07 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6836/1647886755,v1:172.21.15.123:6837/1647886755] up:boot 2023-10-07T23:15:08.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:07 smithi067 ceph-mon[82723]: fsmap 7 up:standby 2023-10-07T23:15:08.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:07 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-07T23:15:08.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:07 smithi086 ceph-mon[122131]: Deploying daemon mds.h on smithi086 2023-10-07T23:15:08.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:07 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6836/1647886755,v1:172.21.15.123:6837/1647886755] up:boot 2023-10-07T23:15:08.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:07 smithi086 ceph-mon[122131]: fsmap 7 up:standby 2023-10-07T23:15:08.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:07 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-07T23:15:08.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:07 smithi123 ceph-mon[122926]: Deploying daemon mds.h on smithi086 2023-10-07T23:15:08.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:07 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6836/1647886755,v1:172.21.15.123:6837/1647886755] up:boot 2023-10-07T23:15:08.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:07 smithi123 ceph-mon[122926]: fsmap 7 up:standby 2023-10-07T23:15:08.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:07 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-07T23:15:08.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:08.697 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:15:08 smithi086 systemd[1]: Starting Ceph mds.h for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 -- 172.21.15.67:0/2342236404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 msgr2=0x7fe9140fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 --2- 172.21.15.67:0/2342236404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe9140fe330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe9080049c0 tx=0x7fe908009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 -- 172.21.15.67:0/2342236404 shutdown_connections 2023-10-07T23:15:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 --2- 172.21.15.67:0/2342236404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9140fe870 0x7fe914105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 --2- 172.21.15.67:0/2342236404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9140fccd0 0x7fe9140fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 --2- 172.21.15.67:0/2342236404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe9140fe330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 -- 172.21.15.67:0/2342236404 >> 172.21.15.67:0/2342236404 conn(0x7fe9140f8430 msgr2=0x7fe9140fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 -- 172.21.15.67:0/2342236404 shutdown_connections 2023-10-07T23:15:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.740+0000 7fe91359e700 1 -- 172.21.15.67:0/2342236404 wait complete. 2023-10-07T23:15:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 Processor -- start 2023-10-07T23:15:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 -- start start 2023-10-07T23:15:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9140fccd0 0x7fe914071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe91406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9140fe870 0x7fe9140704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914070c40 con 0x7fe9140fdee0 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914070db0 con 0x7fe9140fe870 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.741+0000 7fe91359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914117b30 con 0x7fe9140fccd0 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe91406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe91406ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48422/0 (socket says 172.21.15.67:48422) 2023-10-07T23:15:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 -- 172.21.15.67:0/933167645 learned_addr learned my addr 172.21.15.67:0/933167645 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe91259c700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9140fccd0 0x7fe914071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe912d9d700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9140fe870 0x7fe9140704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 -- 172.21.15.67:0/933167645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9140fccd0 msgr2=0x7fe914071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9140fccd0 0x7fe914071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 -- 172.21.15.67:0/933167645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9140fe870 msgr2=0x7fe9140704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9140fe870 0x7fe9140704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 -- 172.21.15.67:0/933167645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908005040 con 0x7fe9140fdee0 2023-10-07T23:15:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe911d9b700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe91406ff90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe908009180 tx=0x7fe908009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.742+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe908020070 con 0x7fe9140fdee0 2023-10-07T23:15:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.743+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe914117d80 con 0x7fe9140fdee0 2023-10-07T23:15:08.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.743+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe908009520 con 0x7fe9140fdee0 2023-10-07T23:15:08.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.743+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800e070 con 0x7fe9140fdee0 2023-10-07T23:15:08.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.743+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe914118210 con 0x7fe9140fdee0 2023-10-07T23:15:08.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.744+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914108530 con 0x7fe9140fdee0 2023-10-07T23:15:08.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.746+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fe908009690 con 0x7fe9140fdee0 2023-10-07T23:15:08.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.748+0000 7fe902ffd700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe8fc09b770 0x7fe8fc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:08.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.748+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fe9080c6480 con 0x7fe9140fdee0 2023-10-07T23:15:08.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.749+0000 7fe91259c700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe8fc09b770 0x7fe8fc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:08.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.749+0000 7fe91259c700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe8fc09b770 0x7fe8fc09dc20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe904009b80 tx=0x7fe90400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:08.751+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fe908092090 con 0x7fe9140fdee0 2023-10-07T23:15:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.002+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe914103be0 con 0x7fe9140fdee0 2023-10-07T23:15:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.002+0000 7fe902ffd700 1 -- 172.21.15.67:0/933167645 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7fe9080956b0 con 0x7fe9140fdee0 2023-10-07T23:15:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:09.004 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-10-07T23:15:09.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe8fc09b770 msgr2=0x7fe8fc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:09.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe8fc09b770 0x7fe8fc09dc20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe904009b80 tx=0x7fe90400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:09.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 msgr2=0x7fe91406ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:09.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe91406ff90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe908009180 tx=0x7fe908009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:09.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 shutdown_connections 2023-10-07T23:15:09.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9140fe870 0x7fe9140704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:09.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe8fc09b770 0x7fe8fc09dc20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:09.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9140fccd0 0x7fe914071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:09.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 --2- 172.21.15.67:0/933167645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9140fdee0 0x7fe91406ff90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:09.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 >> 172.21.15.67:0/933167645 conn(0x7fe9140f8430 msgr2=0x7fe9140fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:09.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 shutdown_connections 2023-10-07T23:15:09.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:09.005+0000 7fe91359e700 1 -- 172.21.15.67:0/933167645 wait complete. 2023-10-07T23:15:09.008 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-07T23:15:09.063 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:08 smithi086 ceph-mon[122131]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:09.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:08 smithi067 ceph-mon[82723]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:09.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:08 smithi123 ceph-mon[122926]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:09.447 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mds[165854]: -- 172.21.15.86:0/176179837 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb47aa09c0 con 0x55cb47a77000 2023-10-07T23:15:09.447 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:15:09 smithi086 conmon[165830]: starting mds.h at 2023-10-07T23:15:09.447 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:15:09 smithi086 bash[165790]: a57542ed04d8000b0a21bd967f24c189af10d008e986f536868beb987b5273ca 2023-10-07T23:15:09.447 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:15:09 smithi086 systemd[1]: Started Ceph mds.h for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/933167645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:10.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:10.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:09 smithi123 ceph-mon[122926]: Deploying daemon mds.i on smithi123 2023-10-07T23:15:10.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/933167645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:10.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:09 smithi067 ceph-mon[82723]: Deploying daemon mds.i on smithi123 2023-10-07T23:15:10.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/933167645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:10.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:10.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:10.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:10.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:10.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:09 smithi086 ceph-mon[122131]: Deploying daemon mds.i on smithi123 2023-10-07T23:15:11.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:11.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:10 smithi123 ceph-mon[122926]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:11.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:10 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] up:boot 2023-10-07T23:15:11.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:10 smithi123 ceph-mon[122926]: fsmap 8 up:standby 2023-10-07T23:15:11.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:10 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-07T23:15:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:10 smithi067 ceph-mon[82723]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:11.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:10 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] up:boot 2023-10-07T23:15:11.323 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:10 smithi067 ceph-mon[82723]: fsmap 8 up:standby 2023-10-07T23:15:11.324 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:10 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-07T23:15:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/2249125710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fcce0 msgr2=0x7fb4dc0fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/2249125710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc0fd0f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80049c0 tx=0x7fb4d8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/2249125710 shutdown_connections 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/2249125710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fe880 0x7fb4dc105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/2249125710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc0fd0f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/2249125710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4dc0fdef0 0x7fb4dc0fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/2249125710 >> 172.21.15.67:0/2249125710 conn(0x7fb4dc0f8460 msgr2=0x7fb4dc0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.320+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/2249125710 shutdown_connections 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.321+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/2249125710 wait complete. 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.321+0000 7fb4e2d9d700 1 Processor -- start 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.321+0000 7fb4e2d9d700 1 -- start start 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc10fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4dc0fdef0 0x7fb4dc110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fe880 0x7fb4dc1171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e2d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc1177e0 con 0x7fb4dc0fe880 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e2d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc117950 con 0x7fb4dc0fdef0 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e2d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc117b10 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc10fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc10fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:38670/0 (socket says 172.21.15.67:38670) 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 -- 172.21.15.67:0/393436956 learned_addr learned my addr 172.21.15.67:0/393436956 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e259c700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fe880 0x7fb4dc1171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e159a700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4dc0fdef0 0x7fb4dc110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 -- 172.21.15.67:0/393436956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4dc0fdef0 msgr2=0x7fb4dc110150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4dc0fdef0 0x7fb4dc110150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 -- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fe880 msgr2=0x7fb4dc1171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fe880 0x7fb4dc1171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 -- 172.21.15.67:0/393436956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d8005040 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.322+0000 7fb4e1d9b700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc10fc10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80137e0 tx=0x7fb4d8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.323+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800e050 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.323+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d80097b0 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.323+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc117d90 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.323+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d801be40 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.323+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc1181a0 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.324+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4dc108520 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.326+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb4d801b4f0 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.327+0000 7fb4d27fc700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4c809b770 0x7fb4c809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.327+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fb4d80c6490 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.327+0000 7fb4e159a700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4c809b770 0x7fb4c809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.328+0000 7fb4e159a700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4c809b770 0x7fb4c809dc20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009b80 tx=0x7fb4cc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.329+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fb4d80920a0 con 0x7fb4dc0fcce0 2023-10-07T23:15:11.390 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:15:11 smithi123 systemd[1]: Starting Ceph mds.i for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:11.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:10 smithi086 ceph-mon[122131]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:11.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:10 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] up:boot 2023-10-07T23:15:11.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:10 smithi086 ceph-mon[122131]: fsmap 8 up:standby 2023-10-07T23:15:11.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:10 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-07T23:15:11.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.581+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4dc118330 con 0x7fb4dc0fcce0 2023-10-07T23:15:12.754 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/393436956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:12.758 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:11 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.582+0000 7fb4d27fc700 1 -- 172.21.15.67:0/393436956 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6559 (secure 0 0 0) 0x7fb4d80956c0 con 0x7fb4dc0fcce0 2023-10-07T23:15:12.759 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:15:11 smithi123 bash[168267]: f6a39fbd68b0ad5f2b0ccec694f26e9b323498bc12476cf7d04ef826463b6fe7 2023-10-07T23:15:12.759 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mds[168331]: -- 172.21.15.123:0/2569804567 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588478fe9c0 con 0x5588478d5000 2023-10-07T23:15:12.759 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:15:11 smithi123 conmon[168307]: starting mds.i at 2023-10-07T23:15:12.759 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:15:11 smithi123 systemd[1]: Started Ceph mds.i for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:12.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/393436956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:12.760 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:11 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4c809b770 msgr2=0x7fb4c809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4c809b770 0x7fb4c809dc20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009b80 tx=0x7fb4cc00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 msgr2=0x7fb4dc10fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc10fc10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80137e0 tx=0x7fb4d8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 shutdown_connections 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4c809b770 0x7fb4c809dc20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4dc0fe880 0x7fb4dc1171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4dc0fcce0 0x7fb4dc10fc10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 --2- 172.21.15.67:0/393436956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4dc0fdef0 0x7fb4dc110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 >> 172.21.15.67:0/393436956 conn(0x7fb4dc0f8460 msgr2=0x7fb4dc103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:12.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 shutdown_connections 2023-10-07T23:15:12.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:11.584+0000 7fb4e2d9d700 1 -- 172.21.15.67:0/393436956 wait complete. 2023-10-07T23:15:12.763 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 9 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/393436956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:12.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:11 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:13.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:12 smithi067 ceph-mon[82723]: Deploying daemon mds.g on smithi067 2023-10-07T23:15:13.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:12 smithi067 ceph-mon[82723]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:13.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:12 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] up:boot 2023-10-07T23:15:13.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:12 smithi067 ceph-mon[82723]: fsmap 9 up:standby 2023-10-07T23:15:13.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:12 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-07T23:15:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:12 smithi123 ceph-mon[122926]: Deploying daemon mds.g on smithi067 2023-10-07T23:15:13.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:12 smithi123 ceph-mon[122926]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:13.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:12 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] up:boot 2023-10-07T23:15:13.958 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:12 smithi123 ceph-mon[122926]: fsmap 9 up:standby 2023-10-07T23:15:13.959 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:12 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-07T23:15:13.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:12 smithi086 ceph-mon[122131]: Deploying daemon mds.g on smithi067 2023-10-07T23:15:13.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:12 smithi086 ceph-mon[122131]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:13.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:12 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] up:boot 2023-10-07T23:15:13.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:12 smithi086 ceph-mon[122131]: fsmap 9 up:standby 2023-10-07T23:15:13.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:12 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-07T23:15:14.254 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:15:13 smithi067 systemd[1]: Starting Ceph mds.g for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:14.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:14.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:14 smithi067 ceph-mon[82723]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:15.047 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:14 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:15.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:15:14 smithi067 bash[139161]: ad638778567cedd9f189b490ad09c7265d5f3b814b874626cb971d6c513dce94 2023-10-07T23:15:15.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:15:14 smithi067 ceph-mds[139225]: -- 172.21.15.67:0/1464012784 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5642c0e489c0 con 0x5642c0e1f000 2023-10-07T23:15:15.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:15:14 smithi067 conmon[139201]: starting mds.g at 2023-10-07T23:15:15.048 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:15:14 smithi067 systemd[1]: Started Ceph mds.g for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.103+0000 7fcaa159d700 1 -- 172.21.15.67:0/72151632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fdf00 msgr2=0x7fca9c0fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.103+0000 7fcaa159d700 1 --2- 172.21.15.67:0/72151632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fdf00 0x7fca9c0fe350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca90004900 tx=0x7fca90009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 -- 172.21.15.67:0/72151632 shutdown_connections 2023-10-07T23:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 --2- 172.21.15.67:0/72151632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 --2- 172.21.15.67:0/72151632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fe890 0x7fca9c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 --2- 172.21.15.67:0/72151632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fdf00 0x7fca9c0fe350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 -- 172.21.15.67:0/72151632 >> 172.21.15.67:0/72151632 conn(0x7fca9c0f8410 msgr2=0x7fca9c0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 -- 172.21.15.67:0/72151632 shutdown_connections 2023-10-07T23:15:15.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 -- 172.21.15.67:0/72151632 wait complete. 2023-10-07T23:15:15.186 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:14 smithi123 ceph-mon[122926]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:15.186 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:14 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.104+0000 7fcaa159d700 1 Processor -- start 2023-10-07T23:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 -- start start 2023-10-07T23:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fdf00 0x7fca9c06ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fe890 0x7fca9c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c070c40 con 0x7fca9c0fdf00 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c070db0 con 0x7fca9c0fccf0 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa159d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c117a70 con 0x7fca9c0fe890 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fca9bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fca9b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fdf00 0x7fca9c06ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fca9bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:51116/0 (socket says 172.21.15.67:51116) 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fca9bfff700 1 -- 172.21.15.67:0/1311308361 learned_addr learned my addr 172.21.15.67:0/1311308361 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.105+0000 7fcaa0d9c700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fe890 0x7fca9c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9bfff700 1 -- 172.21.15.67:0/1311308361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fe890 msgr2=0x7fca9c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9bfff700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fe890 0x7fca9c0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9bfff700 1 -- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fdf00 msgr2=0x7fca9c06ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9bfff700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fdf00 0x7fca9c06ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9bfff700 1 -- 172.21.15.67:0/1311308361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94009940 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9b7fe700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fdf00 0x7fca9c06ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca9bfff700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c071930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00ebf0 tx=0x7fca8c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0047d0 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8c00f040 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca90005040 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00c980 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.106+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c118020 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.107+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c103bb0 con 0x7fca9c0fccf0 2023-10-07T23:15:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.109+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fca8c0090d0 con 0x7fca9c0fccf0 2023-10-07T23:15:15.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.110+0000 7fca98ff9700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca8409b770 0x7fca8409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:15.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.111+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fca8c0c4690 con 0x7fca9c0fccf0 2023-10-07T23:15:15.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.111+0000 7fca9b7fe700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca8409b770 0x7fca8409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:15.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.111+0000 7fca9b7fe700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca8409b770 0x7fca8409dc20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca90004f70 tx=0x7fca90009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:15.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.112+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fca8c04da00 con 0x7fca9c0fccf0 2023-10-07T23:15:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:14 smithi086 ceph-mon[122131]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:14 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:15.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.386+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca9c02b0e0 con 0x7fca9c0fccf0 2023-10-07T23:15:15.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.387+0000 7fca98ff9700 1 -- 172.21.15.67:0/1311308361 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8079 (secure 0 0 0) 0x7fca8c019090 con 0x7fca9c0fccf0 2023-10-07T23:15:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:15.388 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.389+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca8409b770 msgr2=0x7fca8409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.389+0000 7fcaa159d700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca8409b770 0x7fca8409dc20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca90004f70 tx=0x7fca90009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.389+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 msgr2=0x7fca9c071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.389+0000 7fcaa159d700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c071930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00ebf0 tx=0x7fca8c00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 shutdown_connections 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9c0fccf0 0x7fca9c071930 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca8409b770 0x7fca8409dc20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c0fe890 0x7fca9c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 --2- 172.21.15.67:0/1311308361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c0fdf00 0x7fca9c06ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 >> 172.21.15.67:0/1311308361 conn(0x7fca9c0f8410 msgr2=0x7fca9c0fa660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 shutdown_connections 2023-10-07T23:15:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:15.390+0000 7fcaa159d700 1 -- 172.21.15.67:0/1311308361 wait complete. 2023-10-07T23:15:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 11 2023-10-07T23:15:16.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:16.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6838/3904846086,v1:172.21.15.67:6839/3904846086] up:boot 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: fsmap 10 up:standby 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: Deploying daemon mds.a on smithi067 2023-10-07T23:15:16.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:15 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1311308361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6838/3904846086,v1:172.21.15.67:6839/3904846086] up:boot 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: fsmap 10 up:standby 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:16.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: Deploying daemon mds.a on smithi067 2023-10-07T23:15:16.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:15 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1311308361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:16.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:16.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6838/3904846086,v1:172.21.15.67:6839/3904846086] up:boot 2023-10-07T23:15:16.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: fsmap 10 up:standby 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: Deploying daemon mds.a on smithi067 2023-10-07T23:15:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:15 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1311308361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:16.983 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:16 smithi067 ceph-mon[82723]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:17.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:16 smithi123 ceph-mon[122926]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:17.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:16 smithi086 ceph-mon[122131]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:17.254 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:15:16 smithi067 systemd[1]: Starting Ceph mds.a for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:17.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.619+0000 7f9fee39d700 1 -- 172.21.15.67:0/3574927029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 msgr2=0x7f9fe80709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.619+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3574927029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe80709c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80049c0 tx=0x7f9fd8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 -- 172.21.15.67:0/3574927029 shutdown_connections 2023-10-07T23:15:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3574927029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8071bc0 0x7f9fe8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3574927029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fe8104ba0 0x7f9fe8107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3574927029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe80709c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 -- 172.21.15.67:0/3574927029 >> 172.21.15.67:0/3574927029 conn(0x7f9fe80f8420 msgr2=0x7f9fe80fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 -- 172.21.15.67:0/3574927029 shutdown_connections 2023-10-07T23:15:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.620+0000 7f9fee39d700 1 -- 172.21.15.67:0/3574927029 wait complete. 2023-10-07T23:15:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 Processor -- start 2023-10-07T23:15:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 -- start start 2023-10-07T23:15:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe810fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fe8071bc0 0x7f9fe8110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8104ba0 0x7f9fe81172a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81178e0 con 0x7f9fe8070570 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8117a50 con 0x7f9fe8104ba0 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fee39d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8117c10 con 0x7f9fe8071bc0 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fed39b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe810fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fed39b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe810fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35798/0 (socket says 172.21.15.67:35798) 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fed39b700 1 -- 172.21.15.67:0/3229845544 learned_addr learned my addr 172.21.15.67:0/3229845544 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fecb9a700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fe8071bc0 0x7f9fe8110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.621+0000 7f9fedb9c700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8104ba0 0x7f9fe81172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fed39b700 1 -- 172.21.15.67:0/3229845544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fe8071bc0 msgr2=0x7f9fe8110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fed39b700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fe8071bc0 0x7f9fe8110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fed39b700 1 -- 172.21.15.67:0/3229845544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8104ba0 msgr2=0x7f9fe81172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fed39b700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8104ba0 0x7f9fe81172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fed39b700 1 -- 172.21.15.67:0/3229845544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8005040 con 0x7f9fe8070570 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fed39b700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe810fd10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4001e00 tx=0x7f9fe4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4007b30 con 0x7f9fe8070570 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe400ed80 con 0x7f9fe8070570 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8117ef0 con 0x7f9fe8070570 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4009610 con 0x7f9fe8070570 2023-10-07T23:15:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.622+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe81182f0 con 0x7f9fe8070570 2023-10-07T23:15:17.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.624+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe81032f0 con 0x7f9fe8070570 2023-10-07T23:15:17.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.625+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9fe400e8b0 con 0x7f9fe8070570 2023-10-07T23:15:17.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.626+0000 7f9fddffb700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b770 0x7f9fd409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:17.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.626+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f9fe40c46a0 con 0x7f9fe8070570 2023-10-07T23:15:17.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.627+0000 7f9fecb9a700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b770 0x7f9fd409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:17.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.627+0000 7f9fecb9a700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b770 0x7f9fd409dc20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8004890 tx=0x7f9fd8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:17.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.631+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9fe40902b0 con 0x7f9fe8070570 2023-10-07T23:15:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.905+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fe810a760 con 0x7f9fe8070570 2023-10-07T23:15:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.906+0000 7f9fddffb700 1 -- 172.21.15.67:0/3229845544 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8837 (secure 0 0 0) 0x7f9fe40938d0 con 0x7f9fe8070570 2023-10-07T23:15:17.906 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:17.906 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b770 msgr2=0x7f9fd409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b770 0x7f9fd409dc20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8004890 tx=0x7f9fd8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 msgr2=0x7f9fe810fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe810fd10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4001e00 tx=0x7f9fe4009ec0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 shutdown_connections 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe8104ba0 0x7f9fe81172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b770 0x7f9fd409dc20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fe8070570 0x7f9fe810fd10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 --2- 172.21.15.67:0/3229845544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fe8071bc0 0x7f9fe8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:17.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 >> 172.21.15.67:0/3229845544 conn(0x7f9fe80f8420 msgr2=0x7f9fe80fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:17.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 shutdown_connections 2023-10-07T23:15:17.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:17.908+0000 7f9fee39d700 1 -- 172.21.15.67:0/3229845544 wait complete. 2023-10-07T23:15:17.910 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 12 2023-10-07T23:15:18.254 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:15:17 smithi067 bash[139764]: 0aa2dc462957e490fe11aa257c17b4bc4287dd7de04b306843504673aea5b051 2023-10-07T23:15:18.254 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:15:17 smithi067 ceph-mds[139866]: -- 172.21.15.67:0/4198197476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559703d589c0 con 0x559703d2e800 2023-10-07T23:15:18.254 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:15:17 smithi067 conmon[139842]: starting mds.a at 2023-10-07T23:15:18.255 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:15:17 smithi067 systemd[1]: Started Ceph mds.a for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:18.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6840/3010063667,v1:172.21.15.67:6841/3010063667] up:boot 2023-10-07T23:15:18.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: fsmap 11 up:standby 2023-10-07T23:15:18.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-07T23:15:18.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3229845544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:18.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:18 smithi086 ceph-mon[122131]: Deploying daemon mds.b on smithi086 2023-10-07T23:15:19.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6840/3010063667,v1:172.21.15.67:6841/3010063667] up:boot 2023-10-07T23:15:19.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: fsmap 11 up:standby 2023-10-07T23:15:19.141 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-07T23:15:19.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3229845544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:19.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:19.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:19.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:19.142 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:19.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:19.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:19.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:19.143 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:18 smithi123 ceph-mon[122926]: Deploying daemon mds.b on smithi086 2023-10-07T23:15:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6840/3010063667,v1:172.21.15.67:6841/3010063667] up:boot 2023-10-07T23:15:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: fsmap 11 up:standby 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3229845544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:19.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:18 smithi067 ceph-mon[82723]: Deploying daemon mds.b on smithi086 2023-10-07T23:15:19.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 -- 172.21.15.67:0/1866254937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fdef0 msgr2=0x7f53840fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 --2- 172.21.15.67:0/1866254937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fdef0 0x7f53840fe340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53740049c0 tx=0x7f5374009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 -- 172.21.15.67:0/1866254937 shutdown_connections 2023-10-07T23:15:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 --2- 172.21.15.67:0/1866254937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fe880 0x7f5384105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 --2- 172.21.15.67:0/1866254937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f53840fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 --2- 172.21.15.67:0/1866254937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fdef0 0x7f53840fe340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.132+0000 7f538a99d700 1 -- 172.21.15.67:0/1866254937 >> 172.21.15.67:0/1866254937 conn(0x7f53840f8460 msgr2=0x7f53840fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.133+0000 7f538a99d700 1 -- 172.21.15.67:0/1866254937 shutdown_connections 2023-10-07T23:15:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.133+0000 7f538a99d700 1 -- 172.21.15.67:0/1866254937 wait complete. 2023-10-07T23:15:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.133+0000 7f538a99d700 1 Processor -- start 2023-10-07T23:15:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.133+0000 7f538a99d700 1 -- start start 2023-10-07T23:15:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a99d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f538410fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a99d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fdef0 0x7f5384110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a99d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fe880 0x7f53841170f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a99d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5384117700 con 0x7f53840fdef0 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a99d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5384117870 con 0x7f53840fe880 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a99d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5384117a30 con 0x7f53840fcce0 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538999b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f538410fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538999b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f538410fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:35790/0 (socket says 172.21.15.67:35790) 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538999b700 1 -- 172.21.15.67:0/3513018260 learned_addr learned my addr 172.21.15.67:0/3513018260 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538919a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fdef0 0x7f5384110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538a19c700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fe880 0x7f53841170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538919a700 1 -- 172.21.15.67:0/3513018260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 msgr2=0x7f538410fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538919a700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f538410fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538919a700 1 -- 172.21.15.67:0/3513018260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fe880 msgr2=0x7f53841170f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538919a700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fe880 0x7f53841170f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538919a700 1 -- 172.21.15.67:0/3513018260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374005040 con 0x7f53840fdef0 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.134+0000 7f538999b700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f538410fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.135+0000 7f538919a700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fdef0 0x7f5384110060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5374013000 tx=0x7f5374009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.135+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400e040 con 0x7f53840fdef0 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.135+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5384117cb0 con 0x7f53840fdef0 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.135+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5374009670 con 0x7f53840fdef0 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.135+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374013d90 con 0x7f53840fdef0 2023-10-07T23:15:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.135+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5384118110 con 0x7f53840fdef0 2023-10-07T23:15:20.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.136+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53841084d0 con 0x7f53840fdef0 2023-10-07T23:15:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.138+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f53740134b0 con 0x7f53840fdef0 2023-10-07T23:15:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.139+0000 7f537a7fc700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f537009b770 0x7f537009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.139+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f53740c66c0 con 0x7f53840fdef0 2023-10-07T23:15:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.139+0000 7f538999b700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f537009b770 0x7f537009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.140+0000 7f538999b700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f537009b770 0x7f537009dc20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f538000e580 tx=0x7f538000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.142+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f53740922d0 con 0x7f53840fdef0 2023-10-07T23:15:20.194 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:19 smithi086 systemd[1]: Starting Ceph mds.b for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:15:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.392+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53841182d0 con 0x7f53840fdef0 2023-10-07T23:15:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.393+0000 7f537a7fc700 1 -- 172.21.15.67:0/3513018260 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8837 (secure 0 0 0) 0x7f53740958f0 con 0x7f53840fdef0 2023-10-07T23:15:20.394 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:20.394 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f537009b770 msgr2=0x7f537009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f537009b770 0x7f537009dc20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f538000e580 tx=0x7f538000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fdef0 msgr2=0x7f5384110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fdef0 0x7f5384110060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5374013000 tx=0x7f5374009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 shutdown_connections 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53840fe880 0x7f53841170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f537009b770 0x7f537009dc20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53840fcce0 0x7f538410fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 --2- 172.21.15.67:0/3513018260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53840fdef0 0x7f5384110060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:20.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.395+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 >> 172.21.15.67:0/3513018260 conn(0x7f53840f8460 msgr2=0x7f53840fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:20.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.396+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 shutdown_connections 2023-10-07T23:15:20.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:20.396+0000 7f538a99d700 1 -- 172.21.15.67:0/3513018260 wait complete. 2023-10-07T23:15:20.397 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 12 2023-10-07T23:15:20.540 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mds[166423]: -- 172.21.15.86:0/3097256443 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ba6a8c9c0 con 0x559ba6a62800 2023-10-07T23:15:20.540 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:20 smithi086 conmon[166399]: starting mds.b at 2023-10-07T23:15:20.540 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:20 smithi086 bash[166358]: 807db558f59c475a3ef02f64c556f33618a3fb450052ab8700cef9b2138144f9 2023-10-07T23:15:20.947 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:20 smithi086 systemd[1]: Started Ceph mds.b for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:15:21.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3513018260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:20 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:15:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3513018260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:20 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:15:21.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:21.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3513018260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:21.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:21.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:20 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:15:22.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:21 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] up:boot 2023-10-07T23:15:22.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:21 smithi067 ceph-mon[82723]: fsmap 12 up:standby 2023-10-07T23:15:22.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:21 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-07T23:15:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:21 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] up:boot 2023-10-07T23:15:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:21 smithi123 ceph-mon[122926]: fsmap 12 up:standby 2023-10-07T23:15:22.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:21 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-07T23:15:22.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:22.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:21 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] up:boot 2023-10-07T23:15:22.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:21 smithi086 ceph-mon[122131]: fsmap 12 up:standby 2023-10-07T23:15:22.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:21 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-07T23:15:22.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 -- 172.21.15.67:0/108566190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0feb60 msgr2=0x7f50ac105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:22.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 --2- 172.21.15.67:0/108566190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0feb60 0x7f50ac105400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f50a4004ca0 tx=0x7f50a4009ca0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 -- 172.21.15.67:0/108566190 shutdown_connections 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 --2- 172.21.15.67:0/108566190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0feb60 0x7f50ac105400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 --2- 172.21.15.67:0/108566190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 --2- 172.21.15.67:0/108566190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0fe1b0 0x7f50ac0fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 -- 172.21.15.67:0/108566190 >> 172.21.15.67:0/108566190 conn(0x7f50ac0f8660 msgr2=0x7f50ac0faa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 -- 172.21.15.67:0/108566190 shutdown_connections 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.619+0000 7f50b47ff700 1 -- 172.21.15.67:0/108566190 wait complete. 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 Processor -- start 2023-10-07T23:15:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 -- start start 2023-10-07T23:15:22.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac10fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:22.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0fe1b0 0x7f50ac110270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:22.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0feb60 0x7f50ac1172c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:22.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac117900 con 0x7f50ac0fcf00 2023-10-07T23:15:22.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac117a70 con 0x7f50ac0feb60 2023-10-07T23:15:22.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.620+0000 7f50b47ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac117c30 con 0x7f50ac0fe1b0 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b2d9c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0feb60 0x7f50ac1172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac10fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b2d9c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0feb60 0x7f50ac1172c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:56536/0 (socket says 172.21.15.67:56536) 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b1d9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0fe1b0 0x7f50ac110270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac10fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33814/0 (socket says 172.21.15.67:33814) 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b2d9c700 1 -- 172.21.15.67:0/985326675 learned_addr learned my addr 172.21.15.67:0/985326675 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 -- 172.21.15.67:0/985326675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0fe1b0 msgr2=0x7f50ac110270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0fe1b0 0x7f50ac110270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 -- 172.21.15.67:0/985326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0feb60 msgr2=0x7f50ac1172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0feb60 0x7f50ac1172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 -- 172.21.15.67:0/985326675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a8005040 con 0x7f50ac0fcf00 2023-10-07T23:15:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.621+0000 7f50b259b700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac10fd30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009750 tx=0x7f50a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.622+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8007c00 con 0x7f50ac0fcf00 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.622+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a40048f0 con 0x7f50ac0fcf00 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.622+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a800f040 con 0x7f50ac0fcf00 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.622+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8013b80 con 0x7f50ac0fcf00 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.622+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac118160 con 0x7f50ac0fcf00 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.623+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac108740 con 0x7f50ac0fcf00 2023-10-07T23:15:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.625+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f50a800e840 con 0x7f50ac0fcf00 2023-10-07T23:15:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.626+0000 7f50a2ffd700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509809b770 0x7f509809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.626+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f50a80c48e0 con 0x7f50ac0fcf00 2023-10-07T23:15:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.626+0000 7f50b1d9a700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509809b770 0x7f509809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.627+0000 7f50b1d9a700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509809b770 0x7f509809dc20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f509c009b80 tx=0x7f509c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:22.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.630+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f50a80904f0 con 0x7f50ac0fcf00 2023-10-07T23:15:22.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.899+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50ac1088d0 con 0x7f50ac0fcf00 2023-10-07T23:15:22.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.900+0000 7f50a2ffd700 1 -- 172.21.15.67:0/985326675 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9595 (secure 0 0 0) 0x7f50ac1088d0 con 0x7f50ac0fcf00 2023-10-07T23:15:22.900 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:22.900 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24563,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509809b770 msgr2=0x7f509809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509809b770 0x7f509809dc20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f509c009b80 tx=0x7f509c00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 msgr2=0x7f50ac10fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac10fd30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009750 tx=0x7f50a8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 shutdown_connections 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50ac0feb60 0x7f50ac1172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509809b770 0x7f509809dc20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ac0fcf00 0x7f50ac10fd30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 --2- 172.21.15.67:0/985326675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50ac0fe1b0 0x7f50ac110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 >> 172.21.15.67:0/985326675 conn(0x7f50ac0f8660 msgr2=0x7f50ac0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 shutdown_connections 2023-10-07T23:15:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:22.902+0000 7f50b47ff700 1 -- 172.21.15.67:0/985326675 wait complete. 2023-10-07T23:15:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 13 2023-10-07T23:15:22.922 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 -- 172.21.15.67:0/203758972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 msgr2=0x7f4d600fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 --2- 172.21.15.67:0/203758972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d600fd110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d500049c0 tx=0x7f4d50009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 -- 172.21.15.67:0/203758972 shutdown_connections 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 --2- 172.21.15.67:0/203758972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 0x7f4d60105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 --2- 172.21.15.67:0/203758972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d600fd110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 --2- 172.21.15.67:0/203758972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d600fdf10 0x7f4d600fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 -- 172.21.15.67:0/203758972 >> 172.21.15.67:0/203758972 conn(0x7f4d600f8420 msgr2=0x7f4d600fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:23.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 -- 172.21.15.67:0/203758972 shutdown_connections 2023-10-07T23:15:23.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.117+0000 7f4d6559d700 1 -- 172.21.15.67:0/203758972 wait complete. 2023-10-07T23:15:23.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 Processor -- start 2023-10-07T23:15:23.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 -- start start 2023-10-07T23:15:23.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d60071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d600fdf10 0x7f4d6006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 0x7f4d600704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60070c40 con 0x7f4d600fdf10 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60070db0 con 0x7f4d600fe8a0 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d6559d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60117b50 con 0x7f4d600fcd00 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d5ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d60071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d5ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d60071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:43158/0 (socket says 172.21.15.67:43158) 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d5f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d600fdf10 0x7f4d6006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d5ffff700 1 -- 172.21.15.67:0/2002707679 learned_addr learned my addr 172.21.15.67:0/2002707679 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.118+0000 7f4d64d9c700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 0x7f4d600704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d64d9c700 1 -- 172.21.15.67:0/2002707679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 msgr2=0x7f4d60071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d64d9c700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d60071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d64d9c700 1 -- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d600fdf10 msgr2=0x7f4d6006ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d64d9c700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d600fdf10 0x7f4d6006ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d64d9c700 1 -- 172.21.15.67:0/2002707679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50005040 con 0x7f4d600fe8a0 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d64d9c700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 0x7f4d600704d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800ec20 tx=0x7f4d5800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58007b30 con 0x7f4d600fe8a0 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d60117e00 con 0x7f4d600fe8a0 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5800f040 con 0x7f4d600fe8a0 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.119+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800c9f0 con 0x7f4d600fe8a0 2023-10-07T23:15:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.120+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d601182e0 con 0x7f4d600fe8a0 2023-10-07T23:15:23.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.121+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d601042a0 con 0x7f4d600fe8a0 2023-10-07T23:15:23.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.122+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f4d58009090 con 0x7f4d600fe8a0 2023-10-07T23:15:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.124+0000 7f4d5cff9700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d4809b7f0 0x7f4d4809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.124+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f4d580c5150 con 0x7f4d600fe8a0 2023-10-07T23:15:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.124+0000 7f4d5ffff700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d4809b7f0 0x7f4d4809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.125+0000 7f4d5ffff700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d4809b7f0 0x7f4d4809dca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d50004890 tx=0x7f4d50009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:23.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.127+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4d58090d60 con 0x7f4d600fe8a0 2023-10-07T23:15:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:22 smithi067 ceph-mon[82723]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:22 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/985326675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:23.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:22 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:23.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.342+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d60108590 con 0x7f4d600fe8a0 2023-10-07T23:15:23.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.344+0000 7f4d5cff9700 1 -- 172.21.15.67:0/2002707679 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18101 (secure 0 0 0) 0x7f4d58094380 con 0x7f4d600fe8a0 2023-10-07T23:15:23.345 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:23.345 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:14:39.148128+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.346+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d4809b7f0 msgr2=0x7f4d4809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.346+0000 7f4d6559d700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d4809b7f0 0x7f4d4809dca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d50004890 tx=0x7f4d50009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.346+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 msgr2=0x7f4d600704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.346+0000 7f4d6559d700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 0x7f4d600704d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800ec20 tx=0x7f4d5800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 shutdown_connections 2023-10-07T23:15:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d600fe8a0 0x7f4d600704d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d4809b7f0 0x7f4d4809dca0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d600fcd00 0x7f4d60071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 --2- 172.21.15.67:0/2002707679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d600fdf10 0x7f4d6006ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 >> 172.21.15.67:0/2002707679 conn(0x7f4d600f8420 msgr2=0x7f4d60101420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 shutdown_connections 2023-10-07T23:15:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.347+0000 7f4d6559d700 1 -- 172.21.15.67:0/2002707679 wait complete. 2023-10-07T23:15:23.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:15:23.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:15:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:22 smithi123 ceph-mon[122926]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:22 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/985326675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:22 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:22 smithi086 ceph-mon[122131]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:22 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/985326675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:22 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:23.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 -- 172.21.15.67:0/3702088102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 msgr2=0x7f11340fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 --2- 172.21.15.67:0/3702088102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 0x7f11340fe350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1124004900 tx=0x7f1124009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:23.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 -- 172.21.15.67:0/3702088102 shutdown_connections 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 --2- 172.21.15.67:0/3702088102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fe890 0x7f1134105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 --2- 172.21.15.67:0/3702088102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fccf0 0x7f11340fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 --2- 172.21.15.67:0/3702088102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 0x7f11340fe350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 -- 172.21.15.67:0/3702088102 >> 172.21.15.67:0/3702088102 conn(0x7f11340f8450 msgr2=0x7f11340fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 -- 172.21.15.67:0/3702088102 shutdown_connections 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.568+0000 7f113a39d700 1 -- 172.21.15.67:0/3702088102 wait complete. 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 Processor -- start 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 -- start start 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 0x7f113410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 0x7f11341100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fe890 0x7f1134117150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134117760 con 0x7f11340fe890 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341178d0 con 0x7f11340fccf0 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113a39d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134117a90 con 0x7f11340fdf00 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113939b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 0x7f113410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113939b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 0x7f113410fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:56574/0 (socket says 172.21.15.67:56574) 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f113939b700 1 -- 172.21.15.67:0/83419364 learned_addr learned my addr 172.21.15.67:0/83419364 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f1139b9c700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fe890 0x7f1134117150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.569+0000 7f1138b9a700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 0x7f11341100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113939b700 1 -- 172.21.15.67:0/83419364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 msgr2=0x7f11341100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113939b700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 0x7f11341100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113939b700 1 -- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fe890 msgr2=0x7f1134117150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113939b700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fe890 0x7f1134117150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113939b700 1 -- 172.21.15.67:0/83419364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124005040 con 0x7f11340fccf0 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f1139b9c700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fe890 0x7f1134117150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113939b700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 0x7f113410fb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f113000ebc0 tx=0x7f113000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130007cc0 con 0x7f11340fccf0 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1134117d70 con 0x7f11340fccf0 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f113000f040 con 0x7f11340fccf0 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113000c9f0 con 0x7f11340fccf0 2023-10-07T23:15:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.570+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1134118230 con 0x7f11340fccf0 2023-10-07T23:15:23.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.571+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134108560 con 0x7f11340fccf0 2023-10-07T23:15:23.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.573+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f11300090d0 con 0x7f11340fccf0 2023-10-07T23:15:23.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.574+0000 7f1129ffb700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f112009b770 0x7f112009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:23.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.575+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f11300c4690 con 0x7f11340fccf0 2023-10-07T23:15:23.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.575+0000 7f1138b9a700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f112009b770 0x7f112009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:23.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.575+0000 7f1138b9a700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f112009b770 0x7f112009dc20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1124004f70 tx=0x7f1124009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:23.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.576+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f113004da00 con 0x7f11340fccf0 2023-10-07T23:15:23.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.790+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1134118740 con 0x7f11340fccf0 2023-10-07T23:15:23.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.791+0000 7f1129ffb700 1 -- 172.21.15.67:0/83419364 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11300190a0 con 0x7f11340fccf0 2023-10-07T23:15:23.791 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:15:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f112009b770 msgr2=0x7f112009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f112009b770 0x7f112009dc20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1124004f70 tx=0x7f1124009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 msgr2=0x7f113410fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 0x7f113410fb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f113000ebc0 tx=0x7f113000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 shutdown_connections 2023-10-07T23:15:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11340fccf0 0x7f113410fb80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f112009b770 0x7f112009dc20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11340fe890 0x7f1134117150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 --2- 172.21.15.67:0/83419364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11340fdf00 0x7f11341100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 >> 172.21.15.67:0/83419364 conn(0x7f11340f8450 msgr2=0x7f11340fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 shutdown_connections 2023-10-07T23:15:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:23.793+0000 7f113a39d700 1 -- 172.21.15.67:0/83419364 wait complete. 2023-10-07T23:15:23.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T23:15:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 -- 172.21.15.67:0/657637441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fe8a0 msgr2=0x7f54b4105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 --2- 172.21.15.67:0/657637441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fe8a0 0x7f54b4105160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009d60 tx=0x7f54ac004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 -- 172.21.15.67:0/657637441 shutdown_connections 2023-10-07T23:15:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 --2- 172.21.15.67:0/657637441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b40fcd00 0x7f54b40fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 --2- 172.21.15.67:0/657637441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fe8a0 0x7f54b4105160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 --2- 172.21.15.67:0/657637441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fdf10 0x7f54b40fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 -- 172.21.15.67:0/657637441 >> 172.21.15.67:0/657637441 conn(0x7f54b40f8420 msgr2=0x7f54b40fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 -- 172.21.15.67:0/657637441 shutdown_connections 2023-10-07T23:15:24.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.024+0000 7f54bb19d700 1 -- 172.21.15.67:0/657637441 wait complete. 2023-10-07T23:15:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.025+0000 7f54bb19d700 1 Processor -- start 2023-10-07T23:15:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.025+0000 7f54bb19d700 1 -- start start 2023-10-07T23:15:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.025+0000 7f54bb19d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 0x7f54b410d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.025+0000 7f54bb19d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fdf10 0x7f54b410dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54bb19d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b40fe8a0 0x7f54b4114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54bb19d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4115390 con 0x7f54b40fcd00 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54bb19d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4115500 con 0x7f54b40fe8a0 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54bb19d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b41156c0 con 0x7f54b40fdf10 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 0x7f54b410d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 0x7f54b410d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33856/0 (socket says 172.21.15.67:33856) 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 -- 172.21.15.67:0/4114251500 learned_addr learned my addr 172.21.15.67:0/4114251500 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba99c700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b40fe8a0 0x7f54b4114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54b999a700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fdf10 0x7f54b410dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 -- 172.21.15.67:0/4114251500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fdf10 msgr2=0x7f54b410dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fdf10 0x7f54b410dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 -- 172.21.15.67:0/4114251500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b40fe8a0 msgr2=0x7f54b4114d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b40fe8a0 0x7f54b4114d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 -- 172.21.15.67:0/4114251500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac009940 con 0x7f54b40fcd00 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.026+0000 7f54ba19b700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 0x7f54b410d7b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54b000ebf0 tx=0x7f54b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.027+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0007cc0 con 0x7f54b40fcd00 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.027+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b4115940 con 0x7f54b40fcd00 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.027+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b000f040 con 0x7f54b40fcd00 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.027+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000c8f0 con 0x7f54b40fcd00 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.027+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4115f00 con 0x7f54b40fcd00 2023-10-07T23:15:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.028+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b41039e0 con 0x7f54b40fcd00 2023-10-07T23:15:24.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.030+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f54b000cd20 con 0x7f54b40fcd00 2023-10-07T23:15:24.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.031+0000 7f54aaffd700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f54a009b7f0 0x7f54a009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.031+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f54b00c45e0 con 0x7f54b40fcd00 2023-10-07T23:15:24.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.031+0000 7f54b999a700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f54a009b7f0 0x7f54a009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.032+0000 7f54b999a700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f54a009b7f0 0x7f54a009dca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a4004600 tx=0x7f54a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:24.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.035+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f54b004da00 con 0x7f54b40fcd00 2023-10-07T23:15:24.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:23 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2002707679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:24.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:23 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:23 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/83419364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:15:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.254+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f54b4108550 con 0x7f54b40fcd00 2023-10-07T23:15:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.254+0000 7f54aaffd700 1 -- 172.21.15.67:0/4114251500 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f54b00901f0 con 0x7f54b40fcd00 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-07T23:15:24.256 INFO:teuthology.orchestra.run.smithi067.stdout:] 2023-10-07T23:15:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f54a009b7f0 msgr2=0x7f54a009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f54a009b7f0 0x7f54a009dca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a4004600 tx=0x7f54a400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 msgr2=0x7f54b410d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 0x7f54b410d7b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54b000ebf0 tx=0x7f54b000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 shutdown_connections 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54b40fe8a0 0x7f54b4114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f54a009b7f0 0x7f54a009dca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54b40fcd00 0x7f54b410d7b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 --2- 172.21.15.67:0/4114251500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b40fdf10 0x7f54b410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 >> 172.21.15.67:0/4114251500 conn(0x7f54b40f8420 msgr2=0x7f54b40fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 shutdown_connections 2023-10-07T23:15:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.257+0000 7f54bb19d700 1 -- 172.21.15.67:0/4114251500 wait complete. 2023-10-07T23:15:24.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T23:15:24.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T23:15:24.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:23 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2002707679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:24.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:23 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:23 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/83419364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:15:24.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:23 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2002707679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:24.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:23 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:24.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:23 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/83419364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.477+0000 7f288c99d700 1 -- 172.21.15.67:0/277184480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fe8e0 msgr2=0x7f28881051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.477+0000 7f288c99d700 1 --2- 172.21.15.67:0/277184480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fe8e0 0x7f28881051a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2880009d60 tx=0x7f2880004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 -- 172.21.15.67:0/277184480 shutdown_connections 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 --2- 172.21.15.67:0/277184480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fe8e0 0x7f28881051a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 --2- 172.21.15.67:0/277184480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28880fcd40 0x7f28880fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 --2- 172.21.15.67:0/277184480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fdf50 0x7f28880fe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 -- 172.21.15.67:0/277184480 >> 172.21.15.67:0/277184480 conn(0x7f28880f8480 msgr2=0x7f28880fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 -- 172.21.15.67:0/277184480 shutdown_connections 2023-10-07T23:15:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 -- 172.21.15.67:0/277184480 wait complete. 2023-10-07T23:15:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.478+0000 7f288c99d700 1 Processor -- start 2023-10-07T23:15:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 -- start start 2023-10-07T23:15:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fcd40 0x7f288810d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 0x7f288810dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28880fe8e0 0x7f2888114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888115380 con 0x7f28880fdf50 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28881154f0 con 0x7f28880fcd40 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f288c99d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28881156b0 con 0x7f28880fe8e0 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 0x7f288810dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f28877fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fcd40 0x7f288810d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 0x7f288810dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33886/0 (socket says 172.21.15.67:33886) 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 -- 172.21.15.67:0/3783457149 learned_addr learned my addr 172.21.15.67:0/3783457149 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2887fff700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28880fe8e0 0x7f2888114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 -- 172.21.15.67:0/3783457149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28880fe8e0 msgr2=0x7f2888114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28880fe8e0 0x7f2888114d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 -- 172.21.15.67:0/3783457149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fcd40 msgr2=0x7f288810d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fcd40 0x7f288810d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f2886ffd700 1 -- 172.21.15.67:0/3783457149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880009940 con 0x7f28880fdf50 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.479+0000 7f28877fe700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fcd40 0x7f288810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.480+0000 7f2886ffd700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 0x7f288810dce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f287c00ebc0 tx=0x7f287c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.480+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c007cc0 con 0x7f28880fdf50 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.480+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888115930 con 0x7f28880fdf50 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.480+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f287c00f040 con 0x7f28880fdf50 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.480+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c00c840 con 0x7f28880fdf50 2023-10-07T23:15:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.480+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2888115ef0 con 0x7f28880fdf50 2023-10-07T23:15:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.481+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888103950 con 0x7f28880fdf50 2023-10-07T23:15:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.483+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f287c00cc70 con 0x7f28880fdf50 2023-10-07T23:15:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.484+0000 7f286ffff700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f287009b770 0x7f287009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.484+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f287c0c46b0 con 0x7f28880fdf50 2023-10-07T23:15:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.484+0000 7f28877fe700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f287009b770 0x7f287009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.485+0000 7f28877fe700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f287009b770 0x7f287009dc20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2878009b80 tx=0x7f287800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:24.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.488+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f287c04da00 con 0x7f28880fdf50 2023-10-07T23:15:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:24.696+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f28881085b0 con 0x7f28880fdf50 2023-10-07T23:15:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.327+0000 7f286ffff700 1 -- 172.21.15.67:0/3783457149 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f28881085b0 con 0x7f28880fdf50 2023-10-07T23:15:25.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.331+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f287009b770 msgr2=0x7f287009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:25.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.331+0000 7f288c99d700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f287009b770 0x7f287009dc20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2878009b80 tx=0x7f287800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:25.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.331+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 msgr2=0x7f288810dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:25.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.331+0000 7f288c99d700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 0x7f288810dce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f287c00ebc0 tx=0x7f287c00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.332+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 shutdown_connections 2023-10-07T23:15:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.332+0000 7f288c99d700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28880fcd40 0x7f288810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.332+0000 7f288c99d700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f287009b770 0x7f287009dc20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.332+0000 7f288c99d700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28880fe8e0 0x7f2888114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.332+0000 7f288c99d700 1 --2- 172.21.15.67:0/3783457149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28880fdf50 0x7f288810dce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.332+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 >> 172.21.15.67:0/3783457149 conn(0x7f28880f8480 msgr2=0x7f28880fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.333+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 shutdown_connections 2023-10-07T23:15:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.333+0000 7f288c99d700 1 -- 172.21.15.67:0/3783457149 wait complete. 2023-10-07T23:15:25.336 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2023-10-07T23:15:25.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-07T23:15:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4114251500' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-07T23:15:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:25.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:15:25.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:25 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:15:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 -- 172.21.15.67:0/2203674978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc071bc0 msgr2=0x7f09fc070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 --2- 172.21.15.67:0/2203674978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc071bc0 0x7f09fc070030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0049c0 tx=0x7f09ec009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 -- 172.21.15.67:0/2203674978 shutdown_connections 2023-10-07T23:15:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 --2- 172.21.15.67:0/2203674978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc071bc0 0x7f09fc070030 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 --2- 172.21.15.67:0/2203674978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc104ba0 0x7f09fc107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 --2- 172.21.15.67:0/2203674978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09fc070570 0x7f09fc0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 -- 172.21.15.67:0/2203674978 >> 172.21.15.67:0/2203674978 conn(0x7f09fc0f8420 msgr2=0x7f09fc0fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 -- 172.21.15.67:0/2203674978 shutdown_connections 2023-10-07T23:15:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.566+0000 7f09fb59e700 1 -- 172.21.15.67:0/2203674978 wait complete. 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.567+0000 7f09fb59e700 1 Processor -- start 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.567+0000 7f09fb59e700 1 -- start start 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 0x7f09fc10fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09fc071bc0 0x7f09fc110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fb59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc104ba0 0x7f09fc1171c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fb59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc117800 con 0x7f09fc071bc0 2023-10-07T23:15:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fb59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc117970 con 0x7f09fc104ba0 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc117b30 con 0x7f09fc070570 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 0x7f09fc10fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 0x7f09fc10fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:43224/0 (socket says 172.21.15.67:43224) 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc104ba0 0x7f09fc1171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 -- 172.21.15.67:0/1021312093 learned_addr learned my addr 172.21.15.67:0/1021312093 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09f9d9b700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09fc071bc0 0x7f09fc110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 -- 172.21.15.67:0/1021312093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc104ba0 msgr2=0x7f09fc1171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc104ba0 0x7f09fc1171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 -- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09fc071bc0 msgr2=0x7f09fc110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09fc071bc0 0x7f09fc110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fa59c700 1 -- 172.21.15.67:0/1021312093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40048f0 con 0x7f09fc070570 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.568+0000 7f09fad9d700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc104ba0 0x7f09fc1171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.569+0000 7f09fa59c700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 0x7f09fc10fc30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009180 tx=0x7f09ec009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.569+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec020070 con 0x7f09fc070570 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.569+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ec01c430 con 0x7f09fc070570 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.569+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec005040 con 0x7f09fc070570 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.569+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00e070 con 0x7f09fc070570 2023-10-07T23:15:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.569+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc118020 con 0x7f09fc070570 2023-10-07T23:15:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.570+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc1032f0 con 0x7f09fc070570 2023-10-07T23:15:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.572+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f09ec01c5a0 con 0x7f09fc070570 2023-10-07T23:15:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.573+0000 7f09eaffd700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f09e409b770 0x7f09e409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.573+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7493+0+0 (secure 0 0 0) 0x7f09ec0c5de0 con 0x7f09fc070570 2023-10-07T23:15:25.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.573+0000 7f09f9d9b700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f09e409b770 0x7f09e409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:25.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.574+0000 7f09f9d9b700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f09e409b770 0x7f09e409dc20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009b80 tx=0x7f09f000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:25.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.577+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f09ec091880 con 0x7f09fc070570 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4114251500' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:25 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:15:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4114251500' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-07T23:15:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:25.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:15:25.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:25.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:25 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:15:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:25.788+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f09fc10a780 con 0x7f09fc070570 2023-10-07T23:15:26.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.347+0000 7f09eaffd700 1 -- 172.21.15.67:0/1021312093 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f09ec094ea0 con 0x7f09fc070570 2023-10-07T23:15:26.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.353+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f09e409b770 msgr2=0x7f09e409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:26.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.353+0000 7f09fb59e700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f09e409b770 0x7f09e409dc20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009b80 tx=0x7f09f000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:26.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.353+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 msgr2=0x7f09fc10fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:26.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.353+0000 7f09fb59e700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 0x7f09fc10fc30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009180 tx=0x7f09ec009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:26.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.353+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 shutdown_connections 2023-10-07T23:15:26.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.353+0000 7f09fb59e700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09fc104ba0 0x7f09fc1171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.354+0000 7f09fb59e700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f09e409b770 0x7f09e409dc20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.354+0000 7f09fb59e700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09fc070570 0x7f09fc10fc30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.354+0000 7f09fb59e700 1 --2- 172.21.15.67:0/1021312093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09fc071bc0 0x7f09fc110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.354+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 >> 172.21.15.67:0/1021312093 conn(0x7f09fc0f8420 msgr2=0x7f09fc0fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:26.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.355+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 shutdown_connections 2023-10-07T23:15:26.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.355+0000 7f09fb59e700 1 -- 172.21.15.67:0/1021312093 wait complete. 2023-10-07T23:15:26.359 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2023-10-07T23:15:26.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T23:15:26.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 -- 172.21.15.67:0/464492864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54070570 msgr2=0x7f8a540709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:26.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 --2- 172.21.15.67:0/464492864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54070570 0x7f8a540709c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a480049c0 tx=0x7f8a48009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:26.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 -- 172.21.15.67:0/464492864 shutdown_connections 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 --2- 172.21.15.67:0/464492864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a54104ba0 0x7f8a54107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 --2- 172.21.15.67:0/464492864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54071bc0 0x7f8a54070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 --2- 172.21.15.67:0/464492864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54070570 0x7f8a540709c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 -- 172.21.15.67:0/464492864 >> 172.21.15.67:0/464492864 conn(0x7f8a540f8420 msgr2=0x7f8a540fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 -- 172.21.15.67:0/464492864 shutdown_connections 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.583+0000 7f8a5359e700 1 -- 172.21.15.67:0/464492864 wait complete. 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 Processor -- start 2023-10-07T23:15:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 -- start start 2023-10-07T23:15:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54070570 0x7f8a5410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 0x7f8a54110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a54104ba0 0x7f8a54117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a541177d0 con 0x7f8a54104ba0 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54117940 con 0x7f8a54071bc0 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.584+0000 7f8a5359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54117b00 con 0x7f8a54070570 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54070570 0x7f8a5410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 0x7f8a54110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54070570 0x7f8a5410fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:43244/0 (socket says 172.21.15.67:43244) 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a5259c700 1 -- 172.21.15.67:0/2919645433 learned_addr learned my addr 172.21.15.67:0/2919645433 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 0x7f8a54110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:56642/0 (socket says 172.21.15.67:56642) 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a52d9d700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a54104ba0 0x7f8a54117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 -- 172.21.15.67:0/2919645433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54070570 msgr2=0x7f8a5410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54070570 0x7f8a5410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 -- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a54104ba0 msgr2=0x7f8a54117190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a54104ba0 0x7f8a54117190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 -- 172.21.15.67:0/2919645433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44005040 con 0x7f8a54071bc0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a51d9b700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 0x7f8a54110140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a480049c0 tx=0x7f8a48009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800e070 con 0x7f8a54071bc0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a48004160 con 0x7f8a54071bc0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a48005040 con 0x7f8a54071bc0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.585+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a480123c0 con 0x7f8a54071bc0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.586+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a54118050 con 0x7f8a54071bc0 2023-10-07T23:15:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.587+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a541032f0 con 0x7f8a54071bc0 2023-10-07T23:15:26.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.589+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f8a4801c610 con 0x7f8a54071bc0 2023-10-07T23:15:26.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.590+0000 7f8a42ffd700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8a3c09b770 0x7f8a3c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:26.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.590+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7875+0+0 (secure 0 0 0) 0x7f8a480c6430 con 0x7f8a54071bc0 2023-10-07T23:15:26.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.590+0000 7f8a5259c700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8a3c09b770 0x7f8a3c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:26.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.590+0000 7f8a5259c700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8a3c09b770 0x7f8a3c09dc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a541110f0 tx=0x7f8a4400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:26.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.591+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8a48091de0 con 0x7f8a54071bc0 2023-10-07T23:15:26.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3783457149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-07T23:15:26.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3783457149' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-07T23:15:26.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:26 smithi123 ceph-mon[122926]: osdmap e82: 12 total, 12 up, 12 in 2023-10-07T23:15:26.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3783457149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-07T23:15:26.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3783457149' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-07T23:15:26.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:26 smithi086 ceph-mon[122131]: osdmap e82: 12 total, 12 up, 12 in 2023-10-07T23:15:26.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3783457149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-07T23:15:26.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3783457149' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-07T23:15:26.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:26 smithi067 ceph-mon[82723]: osdmap e82: 12 total, 12 up, 12 in 2023-10-07T23:15:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:26.867+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8a5410a730 con 0x7f8a54071bc0 2023-10-07T23:15:27.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.340+0000 7f8a42ffd700 1 -- 172.21.15.67:0/2919645433 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8a48095400 con 0x7f8a54071bc0 2023-10-07T23:15:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.345+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8a3c09b770 msgr2=0x7f8a3c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.346+0000 7f8a5359e700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8a3c09b770 0x7f8a3c09dc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a541110f0 tx=0x7f8a4400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.346+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 msgr2=0x7f8a54110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.346+0000 7f8a5359e700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 0x7f8a54110140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a480049c0 tx=0x7f8a48009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:27.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.346+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 shutdown_connections 2023-10-07T23:15:27.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.346+0000 7f8a5359e700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8a3c09b770 0x7f8a3c09dc20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.347+0000 7f8a5359e700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a54104ba0 0x7f8a54117190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.347+0000 7f8a5359e700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a54070570 0x7f8a5410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.347+0000 7f8a5359e700 1 --2- 172.21.15.67:0/2919645433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a54071bc0 0x7f8a54110140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.347+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 >> 172.21.15.67:0/2919645433 conn(0x7f8a540f8420 msgr2=0x7f8a540fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:27.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.347+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 shutdown_connections 2023-10-07T23:15:27.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.348+0000 7f8a5359e700 1 -- 172.21.15.67:0/2919645433 wait complete. 2023-10-07T23:15:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2023-10-07T23:15:27.365 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-07T23:15:27.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-07T23:15:27.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 -- 172.21.15.67:0/897474411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 msgr2=0x7f219c0fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 --2- 172.21.15.67:0/897474411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 0x7f219c0fe3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f218c004900 tx=0x7f218c009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 -- 172.21.15.67:0/897474411 shutdown_connections 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 --2- 172.21.15.67:0/897474411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fe900 0x7f219c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 --2- 172.21.15.67:0/897474411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219c0fcd60 0x7f219c0fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 --2- 172.21.15.67:0/897474411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 0x7f219c0fe3c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 -- 172.21.15.67:0/897474411 >> 172.21.15.67:0/897474411 conn(0x7f219c0f84c0 msgr2=0x7f219c0fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 -- 172.21.15.67:0/897474411 shutdown_connections 2023-10-07T23:15:27.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.568+0000 7f21a279d700 1 -- 172.21.15.67:0/897474411 wait complete. 2023-10-07T23:15:27.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 Processor -- start 2023-10-07T23:15:27.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 -- start start 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 0x7f219c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 0x7f219c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219c0fe900 0x7f219c114f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c115550 con 0x7f219c0fdf70 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c1156c0 con 0x7f219c0fcd60 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a279d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c115880 con 0x7f219c0fe900 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a179b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 0x7f219c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a179b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 0x7f219c10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:56650/0 (socket says 172.21.15.67:56650) 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a179b700 1 -- 172.21.15.67:0/1947720773 learned_addr learned my addr 172.21.15.67:0/1947720773 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a0f9a700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 0x7f219c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.569+0000 7f21a1f9c700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219c0fe900 0x7f219c114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a179b700 1 -- 172.21.15.67:0/1947720773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219c0fe900 msgr2=0x7f219c114f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a179b700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219c0fe900 0x7f219c114f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a179b700 1 -- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 msgr2=0x7f219c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a179b700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 0x7f219c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a179b700 1 -- 172.21.15.67:0/1947720773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c005040 con 0x7f219c0fcd60 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a179b700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 0x7f219c10d970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f219800ebf0 tx=0x7f219800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198007cc0 con 0x7f219c0fcd60 2023-10-07T23:15:27.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c115b00 con 0x7f219c0fcd60 2023-10-07T23:15:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f219800f040 con 0x7f219c0fcd60 2023-10-07T23:15:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800c8f0 con 0x7f219c0fcd60 2023-10-07T23:15:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.570+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c115ff0 con 0x7f219c0fcd60 2023-10-07T23:15:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.571+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c103bd0 con 0x7f219c0fcd60 2023-10-07T23:15:27.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.573+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f219800cd20 con 0x7f219c0fcd60 2023-10-07T23:15:27.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.575+0000 7f2191ffb700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f218809b770 0x7f218809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:27.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.575+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f21980c4ab0 con 0x7f219c0fcd60 2023-10-07T23:15:27.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.575+0000 7f21a0f9a700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f218809b770 0x7f218809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:27.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.575+0000 7f21a0f9a700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f218809b770 0x7f218809dc20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f218c004f70 tx=0x7f218c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:27.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.576+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f219804da00 con 0x7f219c0fcd60 2023-10-07T23:15:27.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1021312093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: osdmap e83: 12 total, 12 up, 12 in 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2919645433' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: osdmap e84: 12 total, 12 up, 12 in 2023-10-07T23:15:27.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:27 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1021312093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-07T23:15:27.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-07T23:15:27.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: osdmap e83: 12 total, 12 up, 12 in 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2919645433' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: osdmap e84: 12 total, 12 up, 12 in 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1021312093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-07T23:15:27.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-07T23:15:27.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-07T23:15:27.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: osdmap e83: 12 total, 12 up, 12 in 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2919645433' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: osdmap e84: 12 total, 12 up, 12 in 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:27.799+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f219c02b510 con 0x7f219c0fcd60 2023-10-07T23:15:28.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.189+0000 7f2191ffb700 1 -- 172.21.15.67:0/1947720773 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2198019070 con 0x7f219c0fcd60 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f218809b770 msgr2=0x7f218809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f218809b770 0x7f218809dc20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f218c004f70 tx=0x7f218c009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 msgr2=0x7f219c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 0x7f219c10d970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f219800ebf0 tx=0x7f219800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 shutdown_connections 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f219c0fcd60 0x7f219c10d970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f218809b770 0x7f218809dc20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f219c0fe900 0x7f219c114f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 --2- 172.21.15.67:0/1947720773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219c0fdf70 0x7f219c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 >> 172.21.15.67:0/1947720773 conn(0x7f219c0f84c0 msgr2=0x7f219c0fa660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:28.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 shutdown_connections 2023-10-07T23:15:28.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.191+0000 7f21a279d700 1 -- 172.21.15.67:0/1947720773 wait complete. 2023-10-07T23:15:28.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T23:15:28.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.407+0000 7fca1ffff700 1 -- 172.21.15.67:0/3418477459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20070570 msgr2=0x7fca200709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:28.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.407+0000 7fca1ffff700 1 --2- 172.21.15.67:0/3418477459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20070570 0x7fca200709c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca140049c0 tx=0x7fca14009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:28.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 -- 172.21.15.67:0/3418477459 shutdown_connections 2023-10-07T23:15:28.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 --2- 172.21.15.67:0/3418477459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20071bc0 0x7fca20070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 --2- 172.21.15.67:0/3418477459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca20104ba0 0x7fca20107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 --2- 172.21.15.67:0/3418477459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20070570 0x7fca200709c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 -- 172.21.15.67:0/3418477459 >> 172.21.15.67:0/3418477459 conn(0x7fca200f8460 msgr2=0x7fca200fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:28.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 -- 172.21.15.67:0/3418477459 shutdown_connections 2023-10-07T23:15:28.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 -- 172.21.15.67:0/3418477459 wait complete. 2023-10-07T23:15:28.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.408+0000 7fca1ffff700 1 Processor -- start 2023-10-07T23:15:28.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 -- start start 2023-10-07T23:15:28.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca20070570 0x7fca2010fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:28.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 0x7fca20110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:28.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20104ba0 0x7fca20117120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20117730 con 0x7fca20071bc0 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca201178a0 con 0x7fca20104ba0 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20117a60 con 0x7fca20070570 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 0x7fca20110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20104ba0 0x7fca20117120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca20070570 0x7fca2010fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20104ba0 0x7fca20117120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:56658/0 (socket says 172.21.15.67:56658) 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 0x7fca20110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33946/0 (socket says 172.21.15.67:33946) 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.409+0000 7fca1f7fe700 1 -- 172.21.15.67:0/4126749531 learned_addr learned my addr 172.21.15.67:0/4126749531 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1e7fc700 1 -- 172.21.15.67:0/4126749531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca20070570 msgr2=0x7fca2010fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:28.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1e7fc700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca20070570 0x7fca2010fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1e7fc700 1 -- 172.21.15.67:0/4126749531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20104ba0 msgr2=0x7fca20117120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1e7fc700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20104ba0 0x7fca20117120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1e7fc700 1 -- 172.21.15.67:0/4126749531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca14005040 con 0x7fca20071bc0 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1e7fc700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 0x7fca20110090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca14004890 tx=0x7fca14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1400e040 con 0x7fca20071bc0 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20117ce0 con 0x7fca20071bc0 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca14013420 con 0x7fca20071bc0 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1401c660 con 0x7fca20071bc0 2023-10-07T23:15:28.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.410+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca20118140 con 0x7fca20071bc0 2023-10-07T23:15:28.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.411+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca201032f0 con 0x7fca20071bc0 2023-10-07T23:15:28.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.413+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fca14013590 con 0x7fca20071bc0 2023-10-07T23:15:28.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.415+0000 7fca077fe700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca0809b770 0x7fca0809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:28.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.415+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8201+0+0 (secure 0 0 0) 0x7fca140c6af0 con 0x7fca20071bc0 2023-10-07T23:15:28.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.415+0000 7fca1effd700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca0809b770 0x7fca0809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:28.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.416+0000 7fca1effd700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca0809b770 0x7fca0809dc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca10009b80 tx=0x7fca1000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:28.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.417+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fca140922d0 con 0x7fca20071bc0 2023-10-07T23:15:28.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-07T23:15:28.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-07T23:15:28.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-07T23:15:28.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1947720773' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-07T23:15:28.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:28 smithi067 ceph-mon[82723]: osdmap e85: 12 total, 12 up, 12 in 2023-10-07T23:15:28.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:28.623+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fca2010a6e0 con 0x7fca20071bc0 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1947720773' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:15:28.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-07T23:15:28.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:28 smithi123 ceph-mon[122926]: osdmap e85: 12 total, 12 up, 12 in 2023-10-07T23:15:28.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-07T23:15:28.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-07T23:15:28.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-07T23:15:28.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1947720773' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-07T23:15:28.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-07T23:15:28.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:15:28.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:15:28.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:15:28.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-07T23:15:28.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:28 smithi086 ceph-mon[122131]: osdmap e85: 12 total, 12 up, 12 in 2023-10-07T23:15:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: pgmap v284: 97 pgs: 9 creating+peering, 44 active+clean, 44 unknown; 578 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2023-10-07T23:15:29.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-07T23:15:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: Cluster is now healthy 2023-10-07T23:15:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] up:active 2023-10-07T23:15:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:29.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:29 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4126749531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-07T23:15:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: pgmap v284: 97 pgs: 9 creating+peering, 44 active+clean, 44 unknown; 578 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2023-10-07T23:15:29.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-07T23:15:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: Cluster is now healthy 2023-10-07T23:15:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] up:active 2023-10-07T23:15:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:29 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4126749531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-07T23:15:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: pgmap v284: 97 pgs: 9 creating+peering, 44 active+clean, 44 unknown; 578 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2023-10-07T23:15:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-07T23:15:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: Cluster is now healthy 2023-10-07T23:15:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] up:active 2023-10-07T23:15:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:29.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:29.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:29 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4126749531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-07T23:15:30.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.359+0000 7fca077fe700 1 -- 172.21.15.67:0/4126749531 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7fca140958f0 con 0x7fca20071bc0 2023-10-07T23:15:30.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.363+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca0809b770 msgr2=0x7fca0809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:30.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.363+0000 7fca1ffff700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca0809b770 0x7fca0809dc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca10009b80 tx=0x7fca1000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 msgr2=0x7fca20110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 0x7fca20110090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca14004890 tx=0x7fca14009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 shutdown_connections 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca20104ba0 0x7fca20117120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fca0809b770 0x7fca0809dc20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca20070570 0x7fca2010fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 --2- 172.21.15.67:0/4126749531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca20071bc0 0x7fca20110090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 >> 172.21.15.67:0/4126749531 conn(0x7fca200f8460 msgr2=0x7fca200fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 shutdown_connections 2023-10-07T23:15:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.364+0000 7fca1ffff700 1 -- 172.21.15.67:0/4126749531 wait complete. 2023-10-07T23:15:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' created 2023-10-07T23:15:30.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-07T23:15:30.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.579+0000 7f9edf59e700 1 -- 172.21.15.67:0/1149664085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fdf70 msgr2=0x7f9ee00fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:30.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.579+0000 7f9edf59e700 1 --2- 172.21.15.67:0/1149664085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fdf70 0x7f9ee00fe3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40049c0 tx=0x7f9ed4009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.579+0000 7f9edf59e700 1 -- 172.21.15.67:0/1149664085 shutdown_connections 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 --2- 172.21.15.67:0/1149664085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fe900 0x7f9ee01051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 --2- 172.21.15.67:0/1149664085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee00fcd60 0x7f9ee00fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 --2- 172.21.15.67:0/1149664085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fdf70 0x7f9ee00fe3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 -- 172.21.15.67:0/1149664085 >> 172.21.15.67:0/1149664085 conn(0x7f9ee00f84c0 msgr2=0x7f9ee00fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 -- 172.21.15.67:0/1149664085 shutdown_connections 2023-10-07T23:15:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 -- 172.21.15.67:0/1149664085 wait complete. 2023-10-07T23:15:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.580+0000 7f9edf59e700 1 Processor -- start 2023-10-07T23:15:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 -- start start 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 0x7f9ee010d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fdf70 0x7f9ee010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee00fe900 0x7f9ee0114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0115580 con 0x7f9ee00fcd60 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01156f0 con 0x7f9ee00fdf70 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9edf59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01158b0 con 0x7f9ee00fe900 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9ede59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 0x7f9ee010d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9ede59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 0x7f9ee010d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33950/0 (socket says 172.21.15.67:33950) 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9ede59c700 1 -- 172.21.15.67:0/3306295633 learned_addr learned my addr 172.21.15.67:0/3306295633 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9eded9d700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee00fe900 0x7f9ee0114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9eddd9b700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fdf70 0x7f9ee010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.581+0000 7f9ede59c700 1 -- 172.21.15.67:0/3306295633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee00fe900 msgr2=0x7f9ee0114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9ede59c700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee00fe900 0x7f9ee0114f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9ede59c700 1 -- 172.21.15.67:0/3306295633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fdf70 msgr2=0x7f9ee010dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9ede59c700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fdf70 0x7f9ee010dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9ede59c700 1 -- 172.21.15.67:0/3306295633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4005040 con 0x7f9ee00fcd60 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9eddd9b700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fdf70 0x7f9ee010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9ede59c700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 0x7f9ee010d9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000ebc0 tx=0x7f9ed000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0007cc0 con 0x7f9ee00fcd60 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ed000f040 con 0x7f9ee00fcd60 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee0115b30 con 0x7f9ee00fcd60 2023-10-07T23:15:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000c960 con 0x7f9ee00fcd60 2023-10-07T23:15:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.582+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee0115fd0 con 0x7f9ee00fcd60 2023-10-07T23:15:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.583+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0108550 con 0x7f9ee00fcd60 2023-10-07T23:15:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.585+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9ed00090d0 con 0x7f9ee00fcd60 2023-10-07T23:15:30.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.586+0000 7f9eceffd700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9ec809b770 0x7f9ec809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:30.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.587+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8694+0+0 (secure 0 0 0) 0x7f9ed00c5100 con 0x7f9ee00fcd60 2023-10-07T23:15:30.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.587+0000 7f9eddd9b700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9ec809b770 0x7f9ec809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:30.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.587+0000 7f9eddd9b700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9ec809b770 0x7f9ec809dc20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4007f50 tx=0x7f9ed4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:30.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.589+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9ed00906f0 con 0x7f9ee00fcd60 2023-10-07T23:15:30.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:30 smithi123 ceph-mon[122926]: osdmap e86: 12 total, 12 up, 12 in 2023-10-07T23:15:30.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:30 smithi086 ceph-mon[122131]: osdmap e86: 12 total, 12 up, 12 in 2023-10-07T23:15:30.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:30 smithi067 ceph-mon[82723]: osdmap e86: 12 total, 12 up, 12 in 2023-10-07T23:15:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:30.790+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9ee0103ba0 con 0x7f9ee00fcd60 2023-10-07T23:15:31.358 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:30 smithi086 conmon[166399]: 2023-10-07T23:15:30.978+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T23:15:31.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.367+0000 7f9eceffd700 1 -- 172.21.15.67:0/3306295633 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9ed0019020 con 0x7f9ee00fcd60 2023-10-07T23:15:31.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.371+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9ec809b770 msgr2=0x7f9ec809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:31.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.371+0000 7f9edf59e700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9ec809b770 0x7f9ec809dc20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4007f50 tx=0x7f9ed4009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:31.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.371+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 msgr2=0x7f9ee010d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:31.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.371+0000 7f9edf59e700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 0x7f9ee010d9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000ebc0 tx=0x7f9ed000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 shutdown_connections 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9ec809b770 0x7f9ec809dc20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee00fe900 0x7f9ee0114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee00fcd60 0x7f9ee010d9a0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000ebc0 tx=0x7f9ed000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 --2- 172.21.15.67:0/3306295633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee00fdf70 0x7f9ee010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 >> 172.21.15.67:0/3306295633 conn(0x7f9ee00f84c0 msgr2=0x7f9ee00fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:31.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.372+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 shutdown_connections 2023-10-07T23:15:31.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.373+0000 7f9edf59e700 1 -- 172.21.15.67:0/3306295633 wait complete. 2023-10-07T23:15:31.375 INFO:teuthology.orchestra.run.smithi067.stderr:set pool 4 allow_ec_overwrites to true 2023-10-07T23:15:31.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 -- 172.21.15.67:0/2316801008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fdf00 msgr2=0x7f55bc0fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 --2- 172.21.15.67:0/2316801008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fdf00 0x7f55bc0fe350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f55ac004900 tx=0x7f55ac009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 -- 172.21.15.67:0/2316801008 shutdown_connections 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 --2- 172.21.15.67:0/2316801008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fccf0 0x7f55bc0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 --2- 172.21.15.67:0/2316801008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 0x7f55bc105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 --2- 172.21.15.67:0/2316801008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fdf00 0x7f55bc0fe350 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 -- 172.21.15.67:0/2316801008 >> 172.21.15.67:0/2316801008 conn(0x7f55bc0f8410 msgr2=0x7f55bc0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 -- 172.21.15.67:0/2316801008 shutdown_connections 2023-10-07T23:15:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.595+0000 7f55c279d700 1 -- 172.21.15.67:0/2316801008 wait complete. 2023-10-07T23:15:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.596+0000 7f55c279d700 1 Processor -- start 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.596+0000 7f55c279d700 1 -- start start 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.596+0000 7f55c279d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 0x7f55bc10fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.596+0000 7f55c279d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fdf00 0x7f55bc110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.596+0000 7f55c279d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 0x7f55bc117210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c279d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc117860 con 0x7f55bc0fccf0 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c279d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1179d0 con 0x7f55bc0fdf00 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c279d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc117b90 con 0x7f55bc0fe890 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 0x7f55bc10fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 0x7f55bc10fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33964/0 (socket says 172.21.15.67:33964) 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 -- 172.21.15.67:0/1669749976 learned_addr learned my addr 172.21.15.67:0/1669749976 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c0f9a700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fdf00 0x7f55bc110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c1f9c700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 0x7f55bc117210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 -- 172.21.15.67:0/1669749976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 msgr2=0x7f55bc117210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 0x7f55bc117210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 -- 172.21.15.67:0/1669749976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fdf00 msgr2=0x7f55bc110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fdf00 0x7f55bc110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 -- 172.21.15.67:0/1669749976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac005040 con 0x7f55bc0fccf0 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c1f9c700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 0x7f55bc117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c0f9a700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fdf00 0x7f55bc110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.597+0000 7f55c179b700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 0x7f55bc10fc50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55b800ed30 tx=0x7f55b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.598+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800c940 con 0x7f55bc0fccf0 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.598+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b800f040 con 0x7f55bc0fccf0 2023-10-07T23:15:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.598+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc117e70 con 0x7f55bc0fccf0 2023-10-07T23:15:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.598+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8009410 con 0x7f55bc0fccf0 2023-10-07T23:15:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.598+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc1182f0 con 0x7f55bc0fccf0 2023-10-07T23:15:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.599+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc108560 con 0x7f55bc0fccf0 2023-10-07T23:15:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.601+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f55b80079b0 con 0x7f55bc0fccf0 2023-10-07T23:15:31.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.603+0000 7f55b1ffb700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f55a809b770 0x7f55a809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.603+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8694+0+0 (secure 0 0 0) 0x7f55b80c56e0 con 0x7f55bc0fccf0 2023-10-07T23:15:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.603+0000 7f55c0f9a700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f55a809b770 0x7f55a809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.604+0000 7f55c0f9a700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f55a809b770 0x7f55a809dc20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009b80 tx=0x7f55ac009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:31.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.607+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f55b8090cd0 con 0x7f55bc0fccf0 2023-10-07T23:15:31.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:31 smithi123 ceph-mon[122926]: pgmap v287: 97 pgs: 9 creating+peering, 82 active+clean, 6 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2023-10-07T23:15:31.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:31 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4126749531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-07T23:15:31.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:31 smithi123 ceph-mon[122926]: osdmap e87: 12 total, 12 up, 12 in 2023-10-07T23:15:31.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:31 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3306295633' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-07T23:15:31.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:31 smithi086 ceph-mon[122131]: pgmap v287: 97 pgs: 9 creating+peering, 82 active+clean, 6 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2023-10-07T23:15:31.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:31 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4126749531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-07T23:15:31.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:31 smithi086 ceph-mon[122131]: osdmap e87: 12 total, 12 up, 12 in 2023-10-07T23:15:31.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:31 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3306295633' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-07T23:15:31.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:31 smithi067 ceph-mon[82723]: pgmap v287: 97 pgs: 9 creating+peering, 82 active+clean, 6 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2023-10-07T23:15:31.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:31 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4126749531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-07T23:15:31.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:31 smithi067 ceph-mon[82723]: osdmap e87: 12 total, 12 up, 12 in 2023-10-07T23:15:31.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:31 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3306295633' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-07T23:15:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:31.861+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f55bc1187c0 con 0x7f55bc0fccf0 2023-10-07T23:15:32.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.372+0000 7f55b1ffb700 1 -- 172.21.15.67:0/1669749976 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f55b80942f0 con 0x7f55bc0fccf0 2023-10-07T23:15:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.374+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f55a809b770 msgr2=0x7f55a809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.374+0000 7f55c279d700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f55a809b770 0x7f55a809dc20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009b80 tx=0x7f55ac009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.374+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 msgr2=0x7f55bc10fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.374+0000 7f55c279d700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 0x7f55bc10fc50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55b800ed30 tx=0x7f55b800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 shutdown_connections 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f55a809b770 0x7f55a809dc20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55bc0fe890 0x7f55bc117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc0fccf0 0x7f55bc10fc50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 --2- 172.21.15.67:0/1669749976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55bc0fdf00 0x7f55bc110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 >> 172.21.15.67:0/1669749976 conn(0x7f55bc0f8410 msgr2=0x7f55bc0fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 shutdown_connections 2023-10-07T23:15:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.375+0000 7f55c279d700 1 -- 172.21.15.67:0/1669749976 wait complete. 2023-10-07T23:15:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:added data pool 4 to fsmap 2023-10-07T23:15:32.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 -- 172.21.15.67:0/1642258231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 msgr2=0x7f65a8105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/1642258231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8105140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009d60 tx=0x7f65a0004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 -- 172.21.15.67:0/1642258231 shutdown_connections 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/1642258231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8105140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/1642258231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 0x7f65a80fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/1642258231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 0x7f65a80fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 -- 172.21.15.67:0/1642258231 >> 172.21.15.67:0/1642258231 conn(0x7f65a80f8460 msgr2=0x7f65a80fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 -- 172.21.15.67:0/1642258231 shutdown_connections 2023-10-07T23:15:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.586+0000 7f65aeb9d700 1 -- 172.21.15.67:0/1642258231 wait complete. 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 Processor -- start 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 -- start start 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 0x7f65a810f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 0x7f65a810fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8116f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8117580 con 0x7f65a80fe880 2023-10-07T23:15:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81176f0 con 0x7f65a80fdef0 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65aeb9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81178b0 con 0x7f65a80fcce0 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65adb9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 0x7f65a810f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65ad39a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 0x7f65a810fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65ae39c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8116f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65adb9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 0x7f65a810f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:52626/0 (socket says 172.21.15.67:52626) 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.587+0000 7f65adb9b700 1 -- 172.21.15.67:0/4164681874 learned_addr learned my addr 172.21.15.67:0/4164681874 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ad39a700 1 -- 172.21.15.67:0/4164681874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 msgr2=0x7f65a810f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ad39a700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 0x7f65a810f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ad39a700 1 -- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 msgr2=0x7f65a8116f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ad39a700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8116f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ad39a700 1 -- 172.21.15.67:0/4164681874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0009940 con 0x7f65a80fdef0 2023-10-07T23:15:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ae39c700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8116f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65ad39a700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 0x7f65a810fee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6598009fd0 tx=0x7f659800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65980047d0 con 0x7f65a80fdef0 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f659800f040 con 0x7f65a80fdef0 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a8117b30 con 0x7f65a80fdef0 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800c820 con 0x7f65a80fdef0 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.588+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a8118020 con 0x7f65a80fdef0 2023-10-07T23:15:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.589+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a8108500 con 0x7f65a80fdef0 2023-10-07T23:15:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.591+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f659800cc50 con 0x7f65a80fdef0 2023-10-07T23:15:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.592+0000 7f659e7fc700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f659409b770 0x7f659409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.593+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f65980c4b70 con 0x7f65a80fdef0 2023-10-07T23:15:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.593+0000 7f65adb9b700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f659409b770 0x7f659409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.593+0000 7f65adb9b700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f659409b770 0x7f659409dc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009b80 tx=0x7f65a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.595+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f6598090140 con 0x7f65a80fdef0 2023-10-07T23:15:32.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3306295633' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-07T23:15:32.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:32 smithi123 ceph-mon[122926]: osdmap e88: 12 total, 12 up, 12 in 2023-10-07T23:15:32.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1669749976' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-07T23:15:32.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3306295633' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-07T23:15:32.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:32 smithi086 ceph-mon[122131]: osdmap e88: 12 total, 12 up, 12 in 2023-10-07T23:15:32.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1669749976' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-07T23:15:32.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3306295633' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-07T23:15:32.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:32 smithi067 ceph-mon[82723]: osdmap e88: 12 total, 12 up, 12 in 2023-10-07T23:15:32.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1669749976' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-07T23:15:32.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.846+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65a8118210 con 0x7f65a80fdef0 2023-10-07T23:15:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.848+0000 7f659e7fc700 1 -- 172.21.15.67:0/4164681874 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10667 (secure 0 0 0) 0x7f6598019020 con 0x7f65a80fdef0 2023-10-07T23:15:32.848 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:32.849 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:32.363213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-07T23:15:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f659409b770 msgr2=0x7f659409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f659409b770 0x7f659409dc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009b80 tx=0x7f65a400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 msgr2=0x7f65a810fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 0x7f65a810fee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6598009fd0 tx=0x7f659800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 shutdown_connections 2023-10-07T23:15:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f659409b770 0x7f659409dc20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a80fe880 0x7f65a8116f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65a80fcce0 0x7f65a810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 --2- 172.21.15.67:0/4164681874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65a80fdef0 0x7f65a810fee0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 >> 172.21.15.67:0/4164681874 conn(0x7f65a80f8460 msgr2=0x7f65a80fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 shutdown_connections 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:32.850+0000 7f65aeb9d700 1 -- 172.21.15.67:0/4164681874 wait complete. 2023-10-07T23:15:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2023-10-07T23:15:32.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:33.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 -- 172.21.15.67:0/4032885712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 msgr2=0x7f7358105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 --2- 172.21.15.67:0/4032885712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358105150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7350009d60 tx=0x7f7350004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:33.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 -- 172.21.15.67:0/4032885712 shutdown_connections 2023-10-07T23:15:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 --2- 172.21.15.67:0/4032885712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358105150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 --2- 172.21.15.67:0/4032885712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f73580fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 --2- 172.21.15.67:0/4032885712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 0x7f73580fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 -- 172.21.15.67:0/4032885712 >> 172.21.15.67:0/4032885712 conn(0x7f73580f8410 msgr2=0x7f73580fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 -- 172.21.15.67:0/4032885712 shutdown_connections 2023-10-07T23:15:33.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.058+0000 7f735e59d700 1 -- 172.21.15.67:0/4032885712 wait complete. 2023-10-07T23:15:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 Processor -- start 2023-10-07T23:15:33.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 -- start start 2023-10-07T23:15:33.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f735810f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 0x7f735810ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358116f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581175a0 con 0x7f73580fe890 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7358117710 con 0x7f73580fdf00 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735e59d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581178d0 con 0x7f73580fccf0 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735dd9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358116f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735d59b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f735810f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735cd9a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 0x7f735810ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.059+0000 7f735d59b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f735810f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:52642/0 (socket says 172.21.15.67:52642) 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735d59b700 1 -- 172.21.15.67:0/1507148758 learned_addr learned my addr 172.21.15.67:0/1507148758 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735cd9a700 1 -- 172.21.15.67:0/1507148758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 msgr2=0x7f735810f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735cd9a700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f735810f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735cd9a700 1 -- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 msgr2=0x7f7358116f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735cd9a700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358116f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735cd9a700 1 -- 172.21.15.67:0/1507148758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009940 con 0x7f73580fdf00 2023-10-07T23:15:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735d59b700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f735810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735dd9c700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358116f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735cd9a700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 0x7f735810ff00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7348009fd0 tx=0x7f734800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480047d0 con 0x7f73580fdf00 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7358117b50 con 0x7f73580fdf00 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734800f040 con 0x7f73580fdf00 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734800c800 con 0x7f73580fdf00 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.060+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7358118090 con 0x7f73580fdf00 2023-10-07T23:15:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.061+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7358108560 con 0x7f73580fdf00 2023-10-07T23:15:33.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.064+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f734800cc30 con 0x7f73580fdf00 2023-10-07T23:15:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.065+0000 7f734dffb700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f734409b770 0x7f734409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.065+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f73480c4e00 con 0x7f73580fdf00 2023-10-07T23:15:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.066+0000 7f735d59b700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f734409b770 0x7f734409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.066+0000 7f735d59b700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f734409b770 0x7f734409dc20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7354004fd0 tx=0x7f735400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:33.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.067+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f73480903d0 con 0x7f73580fdf00 2023-10-07T23:15:33.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.272+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7358118280 con 0x7f73580fdf00 2023-10-07T23:15:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.274+0000 7f734dffb700 1 -- 172.21.15.67:0/1507148758 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f7348019090 con 0x7f73580fdf00 2023-10-07T23:15:33.275 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:33.275 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:33.277 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f734409b770 msgr2=0x7f734409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f734409b770 0x7f734409dc20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7354004fd0 tx=0x7f735400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 msgr2=0x7f735810ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 0x7f735810ff00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7348009fd0 tx=0x7f734800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 shutdown_connections 2023-10-07T23:15:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f734409b770 0x7f734409dc20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73580fe890 0x7f7358116f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73580fccf0 0x7f735810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 --2- 172.21.15.67:0/1507148758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73580fdf00 0x7f735810ff00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 >> 172.21.15.67:0/1507148758 conn(0x7f73580f8410 msgr2=0x7f73580fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.276+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 shutdown_connections 2023-10-07T23:15:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.277+0000 7f735e59d700 1 -- 172.21.15.67:0/1507148758 wait complete. 2023-10-07T23:15:33.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 -- 172.21.15.67:0/688418107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec1001d0 msgr2=0x7f8aec100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 --2- 172.21.15.67:0/688418107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec1001d0 0x7f8aec100620 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0049c0 tx=0x7f8adc009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 -- 172.21.15.67:0/688418107 shutdown_connections 2023-10-07T23:15:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 --2- 172.21.15.67:0/688418107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec100b60 0x7f8aec107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 --2- 172.21.15.67:0/688418107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec0fefc0 0x7f8aec0ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 --2- 172.21.15.67:0/688418107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec1001d0 0x7f8aec100620 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 -- 172.21.15.67:0/688418107 >> 172.21.15.67:0/688418107 conn(0x7f8aec0fa630 msgr2=0x7f8aec0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 -- 172.21.15.67:0/688418107 shutdown_connections 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.485+0000 7f8af319d700 1 -- 172.21.15.67:0/688418107 wait complete. 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 Processor -- start 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 -- start start 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 0x7f8aec10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec1001d0 0x7f8aec110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 0x7f8aec117110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec117720 con 0x7f8aec100b60 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec117890 con 0x7f8aec0fefc0 2023-10-07T23:15:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af319d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec117a50 con 0x7f8aec1001d0 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af299c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 0x7f8aec117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af219b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 0x7f8aec10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af219b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 0x7f8aec10fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:44732/0 (socket says 172.21.15.67:44732) 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af219b700 1 -- 172.21.15.67:0/500302095 learned_addr learned my addr 172.21.15.67:0/500302095 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af299c700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 0x7f8aec117110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57470/0 (socket says 172.21.15.67:57470) 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.486+0000 7f8af199a700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec1001d0 0x7f8aec110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af219b700 1 -- 172.21.15.67:0/500302095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec1001d0 msgr2=0x7f8aec110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af219b700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec1001d0 0x7f8aec110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af219b700 1 -- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 msgr2=0x7f8aec117110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af219b700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 0x7f8aec117110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af219b700 1 -- 172.21.15.67:0/500302095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc005040 con 0x7f8aec0fefc0 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af299c700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 0x7f8aec117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af219b700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 0x7f8aec10fb40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8002b00 tx=0x7f8ae80046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8007cc0 con 0x7f8aec0fefc0 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec117d30 con 0x7f8aec0fefc0 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae800f040 con 0x7f8aec0fefc0 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.487+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800d9f0 con 0x7f8aec0fefc0 2023-10-07T23:15:33.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.488+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec118160 con 0x7f8aec0fefc0 2023-10-07T23:15:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.489+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec10a7c0 con 0x7f8aec0fefc0 2023-10-07T23:15:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.490+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f8ae8041be0 con 0x7f8aec0fefc0 2023-10-07T23:15:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.492+0000 7f8ae2ffd700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8ad809b770 0x7f8ad809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.492+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f8ae80c4eb0 con 0x7f8aec0fefc0 2023-10-07T23:15:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.492+0000 7f8af199a700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8ad809b770 0x7f8ad809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.492+0000 7f8af199a700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8ad809b770 0x7f8ad809dc20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00b3b0 tx=0x7f8adc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:33.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.494+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8ae8090480 con 0x7f8aec0fefc0 2023-10-07T23:15:33.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: pgmap v290: 129 pgs: 9 creating+peering, 89 active+clean, 31 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2023-10-07T23:15:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-07T23:15:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: osdmap e89: 12 total, 12 up, 12 in 2023-10-07T23:15:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1669749976' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-07T23:15:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4164681874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:33.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:33 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1507148758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:33.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: pgmap v290: 129 pgs: 9 creating+peering, 89 active+clean, 31 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2023-10-07T23:15:33.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-07T23:15:33.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: osdmap e89: 12 total, 12 up, 12 in 2023-10-07T23:15:33.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1669749976' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-07T23:15:33.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:33.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4164681874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:33.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:33 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1507148758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:33.698 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.698+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8aec105e30 con 0x7f8aec0fefc0 2023-10-07T23:15:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.699+0000 7f8ae2ffd700 1 -- 172.21.15.67:0/500302095 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f8ae8093aa0 con 0x7f8aec0fefc0 2023-10-07T23:15:33.700 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:33.700 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8ad809b770 msgr2=0x7f8ad809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8ad809b770 0x7f8ad809dc20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00b3b0 tx=0x7f8adc009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:33.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 msgr2=0x7f8aec10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 0x7f8aec10fb40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8002b00 tx=0x7f8ae80046b0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 shutdown_connections 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aec0fefc0 0x7f8aec10fb40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8ad809b770 0x7f8ad809dc20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aec100b60 0x7f8aec117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 --2- 172.21.15.67:0/500302095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aec1001d0 0x7f8aec110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 >> 172.21.15.67:0/500302095 conn(0x7f8aec0fa630 msgr2=0x7f8aec0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 shutdown_connections 2023-10-07T23:15:33.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.702+0000 7f8af319d700 1 -- 172.21.15.67:0/500302095 wait complete. 2023-10-07T23:15:33.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: pgmap v290: 129 pgs: 9 creating+peering, 89 active+clean, 31 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2023-10-07T23:15:33.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-07T23:15:33.718 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: osdmap e89: 12 total, 12 up, 12 in 2023-10-07T23:15:33.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1669749976' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-07T23:15:33.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:33.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4164681874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:33.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:33 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1507148758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:33.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 -- 172.21.15.67:0/596306288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fe880 msgr2=0x7fd9fc105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 --2- 172.21.15.67:0/596306288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fe880 0x7fd9fc105140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009d60 tx=0x7fd9f4004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 -- 172.21.15.67:0/596306288 shutdown_connections 2023-10-07T23:15:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 --2- 172.21.15.67:0/596306288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fe880 0x7fd9fc105140 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 --2- 172.21.15.67:0/596306288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0fcce0 0x7fd9fc0fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 --2- 172.21.15.67:0/596306288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc0fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 -- 172.21.15.67:0/596306288 >> 172.21.15.67:0/596306288 conn(0x7fd9fc0f8460 msgr2=0x7fd9fc0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 -- 172.21.15.67:0/596306288 shutdown_connections 2023-10-07T23:15:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.930+0000 7fda0179d700 1 -- 172.21.15.67:0/596306288 wait complete. 2023-10-07T23:15:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 Processor -- start 2023-10-07T23:15:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 -- start start 2023-10-07T23:15:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0fcce0 0x7fd9fc10d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc10dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc0fe880 0x7fd9fc114d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc115350 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc1154c0 con 0x7fd9fc0fe880 2023-10-07T23:15:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda0179d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc115680 con 0x7fd9fc0fcce0 2023-10-07T23:15:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fd9fb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc10dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fd9fb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc10dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57484/0 (socket says 172.21.15.67:57484) 2023-10-07T23:15:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fd9fb7fe700 1 -- 172.21.15.67:0/2347495270 learned_addr learned my addr 172.21.15.67:0/2347495270 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fd9fbfff700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0fcce0 0x7fd9fc10d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.931+0000 7fda00f9c700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc0fe880 0x7fd9fc114d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9fb7fe700 1 -- 172.21.15.67:0/2347495270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0fcce0 msgr2=0x7fd9fc10d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9fb7fe700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0fcce0 0x7fd9fc10d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9fb7fe700 1 -- 172.21.15.67:0/2347495270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc0fe880 msgr2=0x7fd9fc114d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:33.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9fb7fe700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc0fe880 0x7fd9fc114d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:33.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9fb7fe700 1 -- 172.21.15.67:0/2347495270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4009940 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9fb7fe700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc10dcb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000ebc0 tx=0x7fd9f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:33.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0007cc0 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc115900 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f000f040 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000c880 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.932+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc115e70 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.933+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc104020 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.935+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd9f000ccb0 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.936+0000 7fd9f8ff9700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd9e409b770 0x7fd9e409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.937+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fd9f00c4ce0 con 0x7fd9fc0fdef0 2023-10-07T23:15:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.937+0000 7fd9fbfff700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd9e409b770 0x7fd9e409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.937+0000 7fd9fbfff700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd9e409b770 0x7fd9e409dc20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009b80 tx=0x7fd9ec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:33.941+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd9f004da00 con 0x7fd9fc0fdef0 2023-10-07T23:15:34.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.189+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9fc1084e0 con 0x7fd9fc0fdef0 2023-10-07T23:15:34.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.189+0000 7fd9f8ff9700 1 -- 172.21.15.67:0/2347495270 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10667 (secure 0 0 0) 0x7fd9fc1084e0 con 0x7fd9fc0fdef0 2023-10-07T23:15:34.190 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:34.191 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:32.363213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2023-10-07T23:15:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd9e409b770 msgr2=0x7fd9e409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd9e409b770 0x7fd9e409dc20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009b80 tx=0x7fd9ec00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 msgr2=0x7fd9fc10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc10dcb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000ebc0 tx=0x7fd9f000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 shutdown_connections 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9fc0fe880 0x7fd9fc114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd9e409b770 0x7fd9e409dc20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0fcce0 0x7fd9fc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 --2- 172.21.15.67:0/2347495270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9fc0fdef0 0x7fd9fc10dcb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 >> 172.21.15.67:0/2347495270 conn(0x7fd9fc0f8460 msgr2=0x7fd9fc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 shutdown_connections 2023-10-07T23:15:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.192+0000 7fda0179d700 1 -- 172.21.15.67:0/2347495270 wait complete. 2023-10-07T23:15:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2023-10-07T23:15:34.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 -- 172.21.15.67:0/2534990208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fe880 msgr2=0x7f8b44105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 --2- 172.21.15.67:0/2534990208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fe880 0x7f8b44105140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009d60 tx=0x7f8b3c004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 -- 172.21.15.67:0/2534990208 shutdown_connections 2023-10-07T23:15:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 --2- 172.21.15.67:0/2534990208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fcce0 0x7f8b440fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 --2- 172.21.15.67:0/2534990208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fe880 0x7f8b44105140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 --2- 172.21.15.67:0/2534990208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b440fdef0 0x7f8b440fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 -- 172.21.15.67:0/2534990208 >> 172.21.15.67:0/2534990208 conn(0x7f8b440f8460 msgr2=0x7f8b440fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 -- 172.21.15.67:0/2534990208 shutdown_connections 2023-10-07T23:15:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.401+0000 7f8b4359e700 1 -- 172.21.15.67:0/2534990208 wait complete. 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.402+0000 7f8b4359e700 1 Processor -- start 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.402+0000 7f8b4359e700 1 -- start start 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b440fcce0 0x7f8b44071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fdef0 0x7f8b4406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 0x7f8b440704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4359e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44070c40 con 0x7f8b440fdef0 2023-10-07T23:15:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4359e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44070db0 con 0x7f8b440fe880 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44117880 con 0x7f8b440fcce0 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b41d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fdef0 0x7f8b4406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 0x7f8b440704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b4259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b440fcce0 0x7f8b44071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 0x7f8b440704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:44744/0 (socket says 172.21.15.67:44744) 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 -- 172.21.15.67:0/1166114691 learned_addr learned my addr 172.21.15.67:0/1166114691 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 -- 172.21.15.67:0/1166114691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b440fcce0 msgr2=0x7f8b44071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b440fcce0 0x7f8b44071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 -- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fdef0 msgr2=0x7f8b4406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fdef0 0x7f8b4406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b42d9d700 1 -- 172.21.15.67:0/1166114691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c009940 con 0x7f8b440fe880 2023-10-07T23:15:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.403+0000 7f8b41d9b700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fdef0 0x7f8b4406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.404+0000 7f8b42d9d700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 0x7f8b440704d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c004b70 tx=0x7f8b3c004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.404+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00e030 con 0x7f8b440fe880 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.404+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3c009520 con 0x7f8b440fe880 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.404+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b44117ad0 con 0x7f8b440fe880 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.404+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c01c690 con 0x7f8b440fe880 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.404+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b44117f80 con 0x7f8b440fe880 2023-10-07T23:15:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.405+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b44104000 con 0x7f8b440fe880 2023-10-07T23:15:34.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.407+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f8b3c012620 con 0x7f8b440fe880 2023-10-07T23:15:34.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.408+0000 7f8b32ffd700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8b2c09b7f0 0x7f8b2c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.408+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f8b3c0c7360 con 0x7f8b440fe880 2023-10-07T23:15:34.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.408+0000 7f8b4259c700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8b2c09b7f0 0x7f8b2c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.409+0000 7f8b4259c700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8b2c09b7f0 0x7f8b2c09dca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009b80 tx=0x7f8b3400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.412+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8b3c092930 con 0x7f8b440fe880 2023-10-07T23:15:34.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:34 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/500302095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:34.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:34 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2347495270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:34.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.615+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b4402b370 con 0x7f8b440fe880 2023-10-07T23:15:34.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.617+0000 7f8b32ffd700 1 -- 172.21.15.67:0/1166114691 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f8b3c017090 con 0x7f8b440fe880 2023-10-07T23:15:34.618 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:34.618 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8b2c09b7f0 msgr2=0x7f8b2c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8b2c09b7f0 0x7f8b2c09dca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009b80 tx=0x7f8b3400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 msgr2=0x7f8b440704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 0x7f8b440704d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c004b70 tx=0x7f8b3c004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 shutdown_connections 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b440fe880 0x7f8b440704d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8b2c09b7f0 0x7f8b2c09dca0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b440fcce0 0x7f8b44071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 --2- 172.21.15.67:0/1166114691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b440fdef0 0x7f8b4406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 >> 172.21.15.67:0/1166114691 conn(0x7f8b440f8460 msgr2=0x7f8b44101180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 shutdown_connections 2023-10-07T23:15:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.619+0000 7f8b4359e700 1 -- 172.21.15.67:0/1166114691 wait complete. 2023-10-07T23:15:34.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T23:15:26.869397+0000', 'modified': '2023-10-07T23:15:32.363213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6839/2147538694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 2147538694}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 2147538694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2023-10-07T23:15:34.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T23:15:34.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-07T23:15:34.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:34 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/500302095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:34.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:34 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2347495270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:34.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:34 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/500302095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:34.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:34 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2347495270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 -- 172.21.15.67:0/2285916976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 msgr2=0x7f9e24070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/2285916976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e24070980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e140049c0 tx=0x7f9e14009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 -- 172.21.15.67:0/2285916976 shutdown_connections 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/2285916976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24104be0 0x7f9e24107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/2285916976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e24071b60 0x7f9e2406ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/2285916976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e24070980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 -- 172.21.15.67:0/2285916976 >> 172.21.15.67:0/2285916976 conn(0x7f9e240f8460 msgr2=0x7f9e240fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 -- 172.21.15.67:0/2285916976 shutdown_connections 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.828+0000 7f9e2a99d700 1 -- 172.21.15.67:0/2285916976 wait complete. 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 Processor -- start 2023-10-07T23:15:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 -- start start 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e2410fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24071b60 0x7f9e24110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e24104be0 0x7f9e241170a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241176b0 con 0x7f9e24070510 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24117820 con 0x7f9e24071b60 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2a99d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241179e0 con 0x7f9e24104be0 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2999b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e2410fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2999b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e2410fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57524/0 (socket says 172.21.15.67:57524) 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.829+0000 7f9e2999b700 1 -- 172.21.15.67:0/4020816918 learned_addr learned my addr 172.21.15.67:0/4020816918 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e24104be0 0x7f9e241170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2919a700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24071b60 0x7f9e24110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2999b700 1 -- 172.21.15.67:0/4020816918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e24104be0 msgr2=0x7f9e241170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2999b700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e24104be0 0x7f9e241170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2999b700 1 -- 172.21.15.67:0/4020816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24071b60 msgr2=0x7f9e24110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2999b700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24071b60 0x7f9e24110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2999b700 1 -- 172.21.15.67:0/4020816918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14005040 con 0x7f9e24070510 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2919a700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24071b60 0x7f9e24110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:15:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2999b700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e2410fad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000ebc0 tx=0x7f9e2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e1a7fc700 1 -- 172.21.15.67:0/4020816918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20007cc0 con 0x7f9e24070510 2023-10-07T23:15:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e24117c60 con 0x7f9e24070510 2023-10-07T23:15:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.830+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e241181a0 con 0x7f9e24070510 2023-10-07T23:15:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.831+0000 7f9e1a7fc700 1 -- 172.21.15.67:0/4020816918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e2000f040 con 0x7f9e24070510 2023-10-07T23:15:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.831+0000 7f9e1a7fc700 1 -- 172.21.15.67:0/4020816918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20009330 con 0x7f9e24070510 2023-10-07T23:15:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.835+0000 7f9e1a7fc700 1 -- 172.21.15.67:0/4020816918 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9e20009490 con 0x7f9e24070510 2023-10-07T23:15:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.836+0000 7f9e3a283b80 1 Processor -- start 2023-10-07T23:15:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.836+0000 7f9e3a283b80 1 -- start start 2023-10-07T23:15:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e52e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e52f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e530140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e483a70 con 0x55bc5e52fcf0 2023-10-07T23:15:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e481270 con 0x55bc5e52e440 2023-10-07T23:15:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e47d060 con 0x55bc5e52ed90 2023-10-07T23:15:34.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e1a7fc700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9e1009b7f0 0x7f9e1009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e52f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2919a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e530140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e52f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:52732/0 (socket says 172.21.15.67:52732) 2023-10-07T23:15:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 -- 172.21.15.67:0/2639560896 learned_addr learned my addr 172.21.15.67:0/2639560896 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e52e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9e1009b7f0 0x7f9e1009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e1a7fc700 1 -- 172.21.15.67:0/4020816918 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f9e200c4b10 con 0x7f9e24070510 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 -- 172.21.15.67:0/2639560896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 msgr2=0x55bc5e52e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e52e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 -- 172.21.15.67:0/2639560896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 msgr2=0x55bc5e530140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e530140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.837+0000 7f9e2999b700 1 -- 172.21.15.67:0/2639560896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5e317270 con 0x55bc5e52ed90 2023-10-07T23:15:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.838+0000 7f9e2999b700 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e52f6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e20090870 tx=0x7f9e200c22f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714b6ad8d77baa2f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.838+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9e1009b7f0 0x7f9e1009dca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c015d60 tx=0x7f9e1c009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.838+0000 7f9e0f7fe700 1 -- 172.21.15.67:0/2639560896 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20094500 con 0x55bc5e52ed90 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.838+0000 7f9e0f7fe700 1 -- 172.21.15.67:0/2639560896 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e20097050 con 0x55bc5e52ed90 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.838+0000 7f9e0f7fe700 1 -- 172.21.15.67:0/2639560896 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e200099b0 con 0x55bc5e52ed90 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 -- 172.21.15.67:0/2639560896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 msgr2=0x55bc5e52f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e52f6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e20090870 tx=0x7f9e200c22f0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 -- 172.21.15.67:0/2639560896 shutdown_connections 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e52e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e530140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/2639560896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e52f6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 -- 172.21.15.67:0/2639560896 >> 172.21.15.67:0/2639560896 conn(0x55bc5e529470 msgr2=0x55bc5e529860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 -- 172.21.15.67:0/2639560896 shutdown_connections 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 -- 172.21.15.67:0/2639560896 wait complete. 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 -- start start 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.839+0000 7f9e3a283b80 1 Processor -- start 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e537d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e538260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52fcf0 0x55bc5e53f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e428890 con 0x55bc5e52ed90 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e443ea0 con 0x55bc5e52e440 2023-10-07T23:15:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e2ffe30 con 0x55bc5e52fcf0 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e3a283b80 10 client.0 _ll_register_callbacks cb 0x55bc5e4f02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e538260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e538260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57534/0 (socket says 172.21.15.67:57534) 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2999b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e537d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 -- 172.21.15.67:0/18628618 learned_addr learned my addr 172.21.15.67:0/18628618 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52fcf0 0x55bc5e53f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 -- 172.21.15.67:0/18628618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52fcf0 msgr2=0x55bc5e53f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52fcf0 0x55bc5e53f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 -- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 msgr2=0x55bc5e537d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.840+0000 7f9e2919a700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e537d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e2919a700 1 -- 172.21.15.67:0/18628618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e241181a0 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e2999b700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e537d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e2919a700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e538260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098c0 tx=0x7f9e14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52594c72fff04d4f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e0cff9700 1 -- 172.21.15.67:0/18628618 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400e040 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc5e521ce0 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e0cff9700 1 -- 172.21.15.67:0/18628618 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f9e14000c90 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e0cff9700 1 -- 172.21.15.67:0/18628618 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1400be90 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e3a283b80 10 client.14622 fetch_fsmap learned FSMap version 17 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.841+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55bc5e2b9fe0 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e0cff9700 1 -- 172.21.15.67:0/18628618 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc5e521ce0 con 0x55bc5e52ed90 2023-10-07T23:15:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e0cff9700 1 -- 172.21.15.67:0/18628618 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) v1 ==== 7468+0+0 (secure 0 0 0) 0x7f9e1401f760 con 0x55bc5e52ed90 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e3a283b80 10 client.14622 fetch_fsmap finished waiting for FSMap version 17 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e3a283b80 10 client.14622 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e3a283b80 20 client.14622 populate_metadata read hostname 'smithi067' 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e542380 0x55bc5e5481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e3a283b80 4 client.14622 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 --> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc5e4feab0 con 0x55bc5e542380 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.842+0000 7f9e2999b700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e542380 0x55bc5e5481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.843+0000 7f9e2999b700 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e542380 0x55bc5e5481d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ddf703fb93e112 server_cookie=71a8aefa2c500ab in_seq=0 out_seq=0 2023-10-07T23:15:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.843+0000 7f9e0cff9700 10 client.14622 ms_handle_connect on v2:172.21.15.86:6838/2147538694 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e0cff9700 1 -- 172.21.15.67:0/18628618 <== mds.0 v2:172.21.15.86:6838/2147538694 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bc5e4feab0 con 0x55bc5e542380 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e0cff9700 10 client.14622 handle_command_reply: tid=0 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 client.14622 shutdown 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 msgr2=0x55bc5e538260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e538260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098c0 tx=0x7f9e14009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 shutdown_connections 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e542380 0x55bc5e5481d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e537d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52fcf0 0x55bc5e53f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e0cff9700 0 client.14622 ms_handle_reset on v2:172.21.15.86:6838/2147538694 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/18628618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e538260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 >> 172.21.15.67:0/18628618 conn(0x55bc5e529470 msgr2=0x55bc5e523f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 shutdown_connections 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.844+0000 7f9e3a283b80 1 -- 172.21.15.67:0/18628618 wait complete. 2023-10-07T23:15:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.845+0000 7f9e3a283b80 20 client.14622 trim_cache size 0 max 16384 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 Processor -- start 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 -- start start 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e535b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e5360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e523670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e4feab0 con 0x55bc5e52fcf0 2023-10-07T23:15:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e52f560 con 0x55bc5e52e440 2023-10-07T23:15:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e448d60 con 0x55bc5e52ed90 2023-10-07T23:15:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e2a19c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e523670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e2a19c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e523670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57544/0 (socket says 172.21.15.67:57544) 2023-10-07T23:15:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.862+0000 7f9e2a19c700 1 -- 172.21.15.67:0/847670972 learned_addr learned my addr 172.21.15.67:0/847670972 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2919a700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e5360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2999b700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e535b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2a19c700 1 -- 172.21.15.67:0/847670972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 msgr2=0x55bc5e5360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e5360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2a19c700 1 -- 172.21.15.67:0/847670972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 msgr2=0x55bc5e535b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e535b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2a19c700 1 -- 172.21.15.67:0/847670972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5e44b780 con 0x55bc5e52fcf0 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2919a700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e5360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2999b700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e535b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e2a19c700 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e523670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c015550 tx=0x7f9e1c008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e75e20240f543c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e0ffff700 1 -- 172.21.15.67:0/847670972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c007c30 con 0x55bc5e52fcf0 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e0ffff700 1 -- 172.21.15.67:0/847670972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c00f040 con 0x55bc5e52fcf0 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.863+0000 7f9e0ffff700 1 -- 172.21.15.67:0/847670972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c01a910 con 0x55bc5e52fcf0 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 -- 172.21.15.67:0/847670972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 msgr2=0x55bc5e523670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e523670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c015550 tx=0x7f9e1c008000 comp rx=0 tx=0).stop 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 -- 172.21.15.67:0/847670972 shutdown_connections 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52e440 0x55bc5e535b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52fcf0 0x55bc5e523670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/847670972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52ed90 0x55bc5e5360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 -- 172.21.15.67:0/847670972 >> 172.21.15.67:0/847670972 conn(0x55bc5e529470 msgr2=0x55bc5e532900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 -- 172.21.15.67:0/847670972 shutdown_connections 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 -- 172.21.15.67:0/847670972 wait complete. 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.864+0000 7f9e3a283b80 1 -- start start 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 Processor -- start 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52e440 0x55bc5e53c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e53c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52fcf0 0x55bc5e594fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e595570 con 0x55bc5e52ed90 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e5956e0 con 0x55bc5e52fcf0 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5e5958a0 con 0x55bc5e52e440 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e3a283b80 10 client.0 _ll_register_callbacks cb 0x55bc5e4f02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:15:34.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e2919a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e53c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e2a19c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52fcf0 0x55bc5e594fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e2919a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e53c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57554/0 (socket says 172.21.15.67:57554) 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e2919a700 1 -- 172.21.15.67:0/271632284 learned_addr learned my addr 172.21.15.67:0/271632284 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.865+0000 7f9e2999b700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52e440 0x55bc5e53c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2919a700 1 -- 172.21.15.67:0/271632284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52e440 msgr2=0x55bc5e53c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2919a700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52e440 0x55bc5e53c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2919a700 1 -- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52fcf0 msgr2=0x55bc5e594fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2919a700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52fcf0 0x55bc5e594fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2919a700 1 -- 172.21.15.67:0/271632284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc5e317270 con 0x55bc5e52ed90 2023-10-07T23:15:34.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2999b700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52e440 0x55bc5e53c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e2919a700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e53c940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009e00 tx=0x7f9e1402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67f0ec7c54f737 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e0cff9700 1 -- 172.21.15.67:0/271632284 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400e030 con 0x55bc5e52ed90 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e20097050 con 0x55bc5e52ed90 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e0cff9700 1 -- 172.21.15.67:0/271632284 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f9e14000bc0 con 0x55bc5e52ed90 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e0cff9700 1 -- 172.21.15.67:0/271632284 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1401f460 con 0x55bc5e52ed90 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e3a283b80 10 client.14628 fetch_fsmap learned FSMap version 17 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.866+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55bc5e2b9fe0 con 0x55bc5e52ed90 2023-10-07T23:15:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e0cff9700 1 -- 172.21.15.67:0/271632284 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e20097050 con 0x55bc5e52ed90 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e0cff9700 1 -- 172.21.15.67:0/271632284 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) v1 ==== 7468+0+0 (secure 0 0 0) 0x7f9e140094b0 con 0x55bc5e52ed90 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e3a283b80 10 client.14628 fetch_fsmap finished waiting for FSMap version 17 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e3a283b80 10 client.14628 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e3a283b80 20 client.14628 populate_metadata read hostname 'smithi067' 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e59a270 0x55bc5e596be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e3a283b80 4 client.14628 mds_command: new command op to 24563 tid=0[{"prefix": "status"}] 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 --> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc5e41fe00 con 0x55bc5e59a270 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.867+0000 7f9e2999b700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e59a270 0x55bc5e596be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e2999b700 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e59a270 0x55bc5e596be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df40c66a2e454fc7 server_cookie=71c71f2ac883c062 in_seq=0 out_seq=0 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e0cff9700 10 client.14628 ms_handle_connect on v2:172.21.15.86:6838/2147538694 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e0cff9700 1 -- 172.21.15.67:0/271632284 <== mds.0 v2:172.21.15.86:6838/2147538694 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55bc5e41fe00 con 0x55bc5e59a270 2023-10-07T23:15:34.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e0cff9700 10 client.14628 handle_command_reply: tid=0 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 client.14628 shutdown 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 msgr2=0x55bc5e53c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e53c940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009e00 tx=0x7f9e1402b040 comp rx=0 tx=0).stop 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 shutdown_connections 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55bc5e59a270 0x55bc5e596be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc5e52fcf0 0x55bc5e594fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc5e52e440 0x55bc5e53c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 --2- 172.21.15.67:0/271632284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bc5e52ed90 0x55bc5e53c940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e0cff9700 0 client.14628 ms_handle_reset on v2:172.21.15.86:6838/2147538694 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 >> 172.21.15.67:0/271632284 conn(0x55bc5e529470 msgr2=0x55bc5e535b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 shutdown_connections 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 1 -- 172.21.15.67:0/271632284 wait complete. 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.868+0000 7f9e3a283b80 20 client.14628 trim_cache size 0 max 16384 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24563, 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:15:34.876 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:15:34.877 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 7.5306995499999996, 2023-10-07T23:15:34.877 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2023-10-07T23:15:34.877 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2023-10-07T23:15:34.877 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:15:34.877 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 14.412355955000001 2023-10-07T23:15:34.877 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9e1009b7f0 msgr2=0x7f9e1009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9e1009b7f0 0x7f9e1009dca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c015d60 tx=0x7f9e1c009f90 comp rx=0 tx=0).stop 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 msgr2=0x7f9e2410fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e2410fad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000ebc0 tx=0x7f9e2000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 shutdown_connections 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9e1009b7f0 0x7f9e1009dca0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e24104be0 0x7f9e241170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e24070510 0x7f9e2410fad0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 --2- 172.21.15.67:0/4020816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e24071b60 0x7f9e24110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:34.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 >> 172.21.15.67:0/4020816918 conn(0x7f9e240f8460 msgr2=0x7f9e24106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:34.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 shutdown_connections 2023-10-07T23:15:34.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:34.869+0000 7f9e2a99d700 1 -- 172.21.15.67:0/4020816918 wait complete. 2023-10-07T23:15:34.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-07T23:15:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:35 smithi123 ceph-mon[122926]: pgmap v292: 129 pgs: 1 creating+peering, 121 active+clean, 7 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-07T23:15:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:35 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1166114691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:35 smithi123 ceph-mon[122926]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-07T23:15:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:35 smithi123 ceph-mon[122926]: Cluster is now healthy 2023-10-07T23:15:35.653 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:15:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:35 smithi086 ceph-mon[122131]: pgmap v292: 129 pgs: 1 creating+peering, 121 active+clean, 7 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-07T23:15:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:35 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1166114691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:35 smithi086 ceph-mon[122131]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-07T23:15:35.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:35 smithi086 ceph-mon[122131]: Cluster is now healthy 2023-10-07T23:15:35.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:35 smithi067 ceph-mon[82723]: pgmap v292: 129 pgs: 1 creating+peering, 121 active+clean, 7 unknown; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2023-10-07T23:15:35.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:35 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1166114691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:35.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:35 smithi067 ceph-mon[82723]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-07T23:15:35.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:35 smithi067 ceph-mon[82723]: Cluster is now healthy 2023-10-07T23:15:37.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 -- 172.21.15.67:0/2658002137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 msgr2=0x7f9914103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:37.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 --2- 172.21.15.67:0/2658002137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 0x7f9914103290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f99040049c0 tx=0x7f9904009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:37.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 -- 172.21.15.67:0/2658002137 shutdown_connections 2023-10-07T23:15:37.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 --2- 172.21.15.67:0/2658002137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 0x7f9914103290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 --2- 172.21.15.67:0/2658002137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066b40 0x7f9914109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 --2- 172.21.15.67:0/2658002137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066190 0x7f9914066600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 -- 172.21.15.67:0/2658002137 >> 172.21.15.67:0/2658002137 conn(0x7f9914073030 msgr2=0x7f9914073430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:37.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 -- 172.21.15.67:0/2658002137 shutdown_connections 2023-10-07T23:15:37.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.065+0000 7f9919275700 1 -- 172.21.15.67:0/2658002137 wait complete. 2023-10-07T23:15:37.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 Processor -- start 2023-10-07T23:15:37.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 -- start start 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066190 0x7f9914113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 0x7f9914114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 0x7f991410dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991410e220 con 0x7f9914066b40 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991410e390 con 0x7f9914102ec0 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f9919275700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991410e550 con 0x7f9914066190 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f99127fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 0x7f9914114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f99127fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 0x7f9914114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57572/0 (socket says 172.21.15.67:57572) 2023-10-07T23:15:37.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.066+0000 7f99127fc700 1 -- 172.21.15.67:0/3364932130 learned_addr learned my addr 172.21.15.67:0/3364932130 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99137fe700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 0x7f991410dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f9912ffd700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066190 0x7f9914113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99127fc700 1 -- 172.21.15.67:0/3364932130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066190 msgr2=0x7f9914113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99127fc700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066190 0x7f9914113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99127fc700 1 -- 172.21.15.67:0/3364932130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 msgr2=0x7f991410dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99127fc700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 0x7f991410dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99127fc700 1 -- 172.21.15.67:0/3364932130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904005040 con 0x7f9914066b40 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f9912ffd700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066190 0x7f9914113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f99127fc700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 0x7f9914114090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f990800ebc0 tx=0x7f990800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908007cc0 con 0x7f9914066b40 2023-10-07T23:15:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991410e7d0 con 0x7f9914066b40 2023-10-07T23:15:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990800f040 con 0x7f9914066b40 2023-10-07T23:15:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.067+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908009520 con 0x7f9914066b40 2023-10-07T23:15:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.068+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991411ab40 con 0x7f9914066b40 2023-10-07T23:15:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.068+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9914063f20 con 0x7f9914066b40 2023-10-07T23:15:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.070+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f990800c630 con 0x7f9914066b40 2023-10-07T23:15:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.072+0000 7f98fb7fe700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f98fc09b770 0x7f98fc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.072+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f99080c4cf0 con 0x7f9914066b40 2023-10-07T23:15:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.072+0000 7f9912ffd700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f98fc09b770 0x7f98fc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:37.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.073+0000 7f9912ffd700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f98fc09b770 0x7f98fc09dc20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9904005010 tx=0x7f9904009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.075+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f990804da00 con 0x7f9914066b40 2023-10-07T23:15:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.320+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f991410ccd0 con 0x7f9914066b40 2023-10-07T23:15:37.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.321+0000 7f98fb7fe700 1 -- 172.21.15.67:0/3364932130 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9908019020 con 0x7f9914066b40 2023-10-07T23:15:37.321 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f98fc09b770 msgr2=0x7f98fc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f98fc09b770 0x7f98fc09dc20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9904005010 tx=0x7f9904009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 msgr2=0x7f9914114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 0x7f9914114090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f990800ebc0 tx=0x7f990800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 shutdown_connections 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9914102ec0 0x7f991410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f98fc09b770 0x7f98fc09dc20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9914066190 0x7f9914113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 --2- 172.21.15.67:0/3364932130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9914066b40 0x7f9914114090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:37.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 >> 172.21.15.67:0/3364932130 conn(0x7f9914073030 msgr2=0x7f99140fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:37.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.323+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 shutdown_connections 2023-10-07T23:15:37.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:37.324+0000 7f9919275700 1 -- 172.21.15.67:0/3364932130 wait complete. 2023-10-07T23:15:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:37 smithi123 ceph-mon[122926]: pgmap v293: 129 pgs: 1 creating+peering, 128 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:37.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:37 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3364932130' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-07T23:15:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:37 smithi086 ceph-mon[122131]: pgmap v293: 129 pgs: 1 creating+peering, 128 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:37.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:37 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3364932130' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-07T23:15:37.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:37 smithi067 ceph-mon[82723]: pgmap v293: 129 pgs: 1 creating+peering, 128 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:37.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:37 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3364932130' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-07T23:15:37.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:15:37.820 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-07T23:15:37.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:37.821 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41772sec preferred_lft 41772sec 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2023-10-07T23:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:15:37.861 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-07T23:15:37.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:37.861 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:15:37.861 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:15:37.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2023-10-07T23:15:37.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2023-10-07T23:15:37.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2023-10-07T23:15:37.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-07T23:15:37.862 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:15:38.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:38.336 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:15:38.402 INFO:teuthology.orchestra.run.smithi067.stdout:1 2023-10-07T23:15:38.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:38.405 DEBUG:teuthology.orchestra.run.smithi067:> route 2023-10-07T23:15:38.484 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2023-10-07T23:15:38.484 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:15:38.484 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T23:15:38.484 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-07T23:15:38.485 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:15:38.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:38.486 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:15:38.486 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:15:38.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T23:15:38.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T23:15:38.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T23:15:38.486 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:15:39.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:39.230 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-07T23:15:39.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:39.267 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2023-10-07T23:15:39.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:39.343 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:15:39.343 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:15:39.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-07T23:15:39.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-07T23:15:39.344 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:15:39.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:39 smithi067 ceph-mon[82723]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:39.518 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-07T23:15:39.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:39.518 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:15:39.518 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:15:39.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-07T23:15:39.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-07T23:15:39.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-07T23:15:39.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-07T23:15:39.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-07T23:15:39.519 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:15:39.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:39 smithi123 ceph-mon[122926]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:39.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:39 smithi086 ceph-mon[122131]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:39.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:39.784 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:15:39.784 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:15:39.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2023-10-07T23:15:39.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2023-10-07T23:15:39.785 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:15:39.945 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-07T23:15:39.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:15:39.946 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:39.982 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-07T23:15:39.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:15:39.983 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:40.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2023-10-07T23:15:40.163 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:15:40.163 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:15:40.231 INFO:teuthology.orchestra.run.smithi067.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:15:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:15:40.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:40.234 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2023-10-07T23:15:40.270 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:15:40.271 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-07T23:15:40.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2023-10-07T23:15:40.380 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:15:40.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:15:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:15:40.446 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:15:40.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:40.446 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2023-10-07T23:15:40.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:2023-10-07T23:15:40.618+0000 7ff64de52540 -1 init, newargv = 0x55647bd5b2d0 newargc=15 2023-10-07T23:15:40.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[142961]: starting ceph client 2023-10-07T23:15:40.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[142961]: starting fuse 2023-10-07T23:15:41.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2023-10-07T23:15:41.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:41 smithi123 ceph-mon[122926]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:41.646 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:41 smithi067 ceph-mon[82723]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:41.657 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:15:41.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:15:41.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:41 smithi086 ceph-mon[122131]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:41.721 INFO:teuthology.orchestra.run.smithi067.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:15:41.723 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:15:41.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:41.724 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2023-10-07T23:15:41.762 INFO:teuthology.orchestra.run.smithi067.stdout:121 2023-10-07T23:15:41.762 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-10-07T23:15:41.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:15:41.762 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2023-10-07T23:15:41.762 DEBUG:teuthology.orchestra.run.smithi067:> import re 2023-10-07T23:15:41.762 DEBUG:teuthology.orchestra.run.smithi067:> import os 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2023-10-07T23:15:41.763 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> return f 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.admin")) 2023-10-07T23:15:41.764 DEBUG:teuthology.orchestra.run.smithi067:> ' 2023-10-07T23:15:41.941 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.admin.142961.asok 2023-10-07T23:15:42.118 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.142961.asok 2023-10-07T23:15:42.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:42.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.142961.asok status 2023-10-07T23:15:42.336 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:15:42.336 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2023-10-07T23:15:42.336 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:15:42.336 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:15:42.336 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "admin", 2023-10-07T23:15:42.336 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "142961", 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 14640, 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2023-10-07T23:15:42.337 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 14640 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3618675929 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3618675929 2023-10-07T23:15:42.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.14640 192.168.0.1:0/3618675929", 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/3618675929", 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 17, 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 89, 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2023-10-07T23:15:42.339 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:15:42.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:42.350 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2023-10-07T23:15:42.350 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:42.390 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2023-10-07T23:15:42.390 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:42.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:42.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:42.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:15:42.495 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-07T23:15:42.604 INFO:teuthology.orchestra.run.smithi067.stdout:# file: . 2023-10-07T23:15:42.604 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-07T23:15:42.604 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:42.605 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2023-10-07T23:15:42.605 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:15:42.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-07T23:15:42.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[142961]: fuse finished with error 0 and tester_r 0 2023-10-07T23:15:42.673 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T23:15:43.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:43 smithi123 ceph-mon[122926]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:43.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:43 smithi086 ceph-mon[122131]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:43.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:43 smithi067 ceph-mon[82723]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-10-07T23:15:45.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:44 smithi123 ceph-mon[122926]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 442 B/s wr, 0 op/s 2023-10-07T23:15:45.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:44 smithi086 ceph-mon[122131]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 442 B/s wr, 0 op/s 2023-10-07T23:15:45.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:44 smithi067 ceph-mon[82723]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 442 B/s wr, 0 op/s 2023-10-07T23:15:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:46 smithi067 ceph-mon[82723]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:47.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:46 smithi123 ceph-mon[122926]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:47.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:46 smithi086 ceph-mon[122131]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:48.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T23:15:48.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:15:48.775 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-07T23:15:48.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 -- 172.21.15.67:0/1059248838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fdef0 msgr2=0x7f36d00fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 --2- 172.21.15.67:0/1059248838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fdef0 0x7f36d00fe340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f36c00049c0 tx=0x7f36c0009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 -- 172.21.15.67:0/1059248838 shutdown_connections 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 --2- 172.21.15.67:0/1059248838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d00fe880 0x7f36d0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 --2- 172.21.15.67:0/1059248838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d00fcce0 0x7f36d00fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 --2- 172.21.15.67:0/1059248838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fdef0 0x7f36d00fe340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 -- 172.21.15.67:0/1059248838 >> 172.21.15.67:0/1059248838 conn(0x7f36d00f8460 msgr2=0x7f36d00fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:49.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 -- 172.21.15.67:0/1059248838 shutdown_connections 2023-10-07T23:15:49.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.028+0000 7f36d759d700 1 -- 172.21.15.67:0/1059248838 wait complete. 2023-10-07T23:15:49.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 Processor -- start 2023-10-07T23:15:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 -- start start 2023-10-07T23:15:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 0x7f36d010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d00fdef0 0x7f36d010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d00fe880 0x7f36d0114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0115500 con 0x7f36d00fcce0 2023-10-07T23:15:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0115670 con 0x7f36d00fe880 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d759d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0115830 con 0x7f36d00fdef0 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d659b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 0x7f36d010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d659b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 0x7f36d010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52816/0 (socket says 172.21.15.67:52816) 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d659b700 1 -- 172.21.15.67:0/3057887261 learned_addr learned my addr 172.21.15.67:0/3057887261 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d6d9c700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d00fe880 0x7f36d0114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.029+0000 7f36d5d9a700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d00fdef0 0x7f36d010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d659b700 1 -- 172.21.15.67:0/3057887261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d00fdef0 msgr2=0x7f36d010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d659b700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d00fdef0 0x7f36d010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d659b700 1 -- 172.21.15.67:0/3057887261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d00fe880 msgr2=0x7f36d0114ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d659b700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d00fe880 0x7f36d0114ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d659b700 1 -- 172.21.15.67:0/3057887261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0005040 con 0x7f36d00fcce0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d659b700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 0x7f36d010d920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00ebf0 tx=0x7f36cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc007cc0 con 0x7f36d00fcce0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d0115ab0 con 0x7f36d00fcce0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc00f040 con 0x7f36d00fcce0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00c8f0 con 0x7f36d00fcce0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.030+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d0115ff0 con 0x7f36d00fcce0 2023-10-07T23:15:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.031+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d0103b50 con 0x7f36d00fcce0 2023-10-07T23:15:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.033+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f36cc00cd20 con 0x7f36d00fcce0 2023-10-07T23:15:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.034+0000 7f36c6ffd700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f36bc09b770 0x7f36bc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.035+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f36cc0c4b80 con 0x7f36d00fcce0 2023-10-07T23:15:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.035+0000 7f36d5d9a700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f36bc09b770 0x7f36bc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.035+0000 7f36d5d9a700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f36bc09b770 0x7f36bc09dc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f36c0007f50 tx=0x7f36c0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:49.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.038+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f36cc04da00 con 0x7f36d00fcce0 2023-10-07T23:15:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.244+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36d0108520 con 0x7f36d00fcce0 2023-10-07T23:15:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.245+0000 7f36c6ffd700 1 -- 172.21.15.67:0/3057887261 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f36cc019020 con 0x7f36d00fcce0 2023-10-07T23:15:49.246 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:49.246 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f36bc09b770 msgr2=0x7f36bc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f36bc09b770 0x7f36bc09dc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f36c0007f50 tx=0x7f36c0009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 msgr2=0x7f36d010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 0x7f36d010d920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00ebf0 tx=0x7f36cc00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 shutdown_connections 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36d00fe880 0x7f36d0114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.247+0000 7f36d759d700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f36bc09b770 0x7f36bc09dc20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.248+0000 7f36d759d700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36d00fcce0 0x7f36d010d920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.248+0000 7f36d759d700 1 --2- 172.21.15.67:0/3057887261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36d00fdef0 0x7f36d010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.248+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 >> 172.21.15.67:0/3057887261 conn(0x7f36d00f8460 msgr2=0x7f36d00fa630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.248+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 shutdown_connections 2023-10-07T23:15:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.248+0000 7f36d759d700 1 -- 172.21.15.67:0/3057887261 wait complete. 2023-10-07T23:15:49.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:48 smithi067 ceph-mon[82723]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:49.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:48 smithi123 ceph-mon[122926]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:48 smithi086 ceph-mon[122131]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 -- 172.21.15.67:0/1024503068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 msgr2=0x7f277c107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 --2- 172.21.15.67:0/1024503068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c107000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2774009d60 tx=0x7f2774004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 -- 172.21.15.67:0/1024503068 shutdown_connections 2023-10-07T23:15:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 --2- 172.21.15.67:0/1024503068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c071bc0 0x7f277c070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 --2- 172.21.15.67:0/1024503068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c107000 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 --2- 172.21.15.67:0/1024503068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c070570 0x7f277c0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.455+0000 7f278359d700 1 -- 172.21.15.67:0/1024503068 >> 172.21.15.67:0/1024503068 conn(0x7f277c0f8460 msgr2=0x7f277c0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:49.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.456+0000 7f278359d700 1 -- 172.21.15.67:0/1024503068 shutdown_connections 2023-10-07T23:15:49.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.456+0000 7f278359d700 1 -- 172.21.15.67:0/1024503068 wait complete. 2023-10-07T23:15:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.456+0000 7f278359d700 1 Processor -- start 2023-10-07T23:15:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.456+0000 7f278359d700 1 -- start start 2023-10-07T23:15:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278359d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 0x7f277c10fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278359d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 0x7f277c10fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278359d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c117080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278359d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c117690 con 0x7f277c104b80 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278359d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c117800 con 0x7f277c070570 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278359d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1179c0 con 0x7f277c071bc0 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2782d9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c117080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2781d9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 0x7f277c10fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 0x7f277c10fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2781d9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 0x7f277c10fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34244/0 (socket says 172.21.15.67:34244) 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2782d9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c117080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52824/0 (socket says 172.21.15.67:52824) 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 0x7f277c10fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:42024/0 (socket says 172.21.15.67:42024) 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2782d9c700 1 -- 172.21.15.67:0/3214144761 learned_addr learned my addr 172.21.15.67:0/3214144761 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 -- 172.21.15.67:0/3214144761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 msgr2=0x7f277c10fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 0x7f277c10fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 -- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 msgr2=0x7f277c117080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c117080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f278259b700 1 -- 172.21.15.67:0/3214144761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778005040 con 0x7f277c070570 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2781d9a700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 0x7f277c10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.457+0000 7f2782d9c700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.458+0000 7f278259b700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 0x7f277c10fab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2778009520 tx=0x7f2778009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.458+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778012570 con 0x7f277c070570 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.458+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2774009940 con 0x7f277c070570 2023-10-07T23:15:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.458+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277800f040 con 0x7f277c070570 2023-10-07T23:15:49.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.458+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277801c9b0 con 0x7f277c070570 2023-10-07T23:15:49.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.458+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c118050 con 0x7f277c070570 2023-10-07T23:15:49.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.459+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c1032d0 con 0x7f277c070570 2023-10-07T23:15:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.461+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f27780128e0 con 0x7f277c070570 2023-10-07T23:15:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.462+0000 7f2772ffd700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f276809b7f0 0x7f276809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.462+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f27780c4ff0 con 0x7f277c070570 2023-10-07T23:15:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.462+0000 7f2781d9a700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f276809b7f0 0x7f276809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.463+0000 7f2781d9a700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f276809b7f0 0x7f276809dca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f276c004fd0 tx=0x7f276c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.464+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2778090690 con 0x7f277c070570 2023-10-07T23:15:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.669+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f277c02b330 con 0x7f277c070570 2023-10-07T23:15:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.671+0000 7f2772ffd700 1 -- 172.21.15.67:0/3214144761 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f2778093cb0 con 0x7f277c070570 2023-10-07T23:15:49.672 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:49.672 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.673+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f276809b7f0 msgr2=0x7f276809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.673+0000 7f278359d700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f276809b7f0 0x7f276809dca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f276c004fd0 tx=0x7f276c00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.673+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 msgr2=0x7f277c10fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.673+0000 7f278359d700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 0x7f277c10fab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2778009520 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.673+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 shutdown_connections 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.673+0000 7f278359d700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f277c070570 0x7f277c10fab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.674+0000 7f278359d700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f276809b7f0 0x7f276809dca0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.674+0000 7f278359d700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f277c104b80 0x7f277c117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.674+0000 7f278359d700 1 --2- 172.21.15.67:0/3214144761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c071bc0 0x7f277c10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.674+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 >> 172.21.15.67:0/3214144761 conn(0x7f277c0f8460 msgr2=0x7f277c0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.674+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 shutdown_connections 2023-10-07T23:15:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.674+0000 7f278359d700 1 -- 172.21.15.67:0/3214144761 wait complete. 2023-10-07T23:15:49.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T23:15:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.884+0000 7f132359e700 1 -- 172.21.15.67:0/1528215542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fdf00 msgr2=0x7f13240fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.884+0000 7f132359e700 1 --2- 172.21.15.67:0/1528215542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fdf00 0x7f13240fe350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13180049c0 tx=0x7f1318009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 -- 172.21.15.67:0/1528215542 shutdown_connections 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 --2- 172.21.15.67:0/1528215542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13240fccf0 0x7f13240fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 --2- 172.21.15.67:0/1528215542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13240fe890 0x7f1324105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 --2- 172.21.15.67:0/1528215542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fdf00 0x7f13240fe350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 -- 172.21.15.67:0/1528215542 >> 172.21.15.67:0/1528215542 conn(0x7f13240f8450 msgr2=0x7f13240fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 -- 172.21.15.67:0/1528215542 shutdown_connections 2023-10-07T23:15:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 -- 172.21.15.67:0/1528215542 wait complete. 2023-10-07T23:15:49.888 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.885+0000 7f132359e700 1 Processor -- start 2023-10-07T23:15:49.888 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 -- start start 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 0x7f1324071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13240fdf00 0x7f132406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13240fe890 0x7f13240704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1324070c40 con 0x7f13240fccf0 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1324070db0 con 0x7f13240fdf00 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1324117b10 con 0x7f13240fe890 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 0x7f1324071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 0x7f1324071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52850/0 (socket says 172.21.15.67:52850) 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f132259c700 1 -- 172.21.15.67:0/1041866061 learned_addr learned my addr 172.21.15.67:0/1041866061 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f1321d9b700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13240fdf00 0x7f132406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.886+0000 7f1322d9d700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13240fe890 0x7f13240704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132259c700 1 -- 172.21.15.67:0/1041866061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13240fe890 msgr2=0x7f13240704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132259c700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13240fe890 0x7f13240704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132259c700 1 -- 172.21.15.67:0/1041866061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13240fdf00 msgr2=0x7f132406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132259c700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13240fdf00 0x7f132406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132259c700 1 -- 172.21.15.67:0/1041866061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318005040 con 0x7f13240fccf0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132259c700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 0x7f1324071930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f131400ebf0 tx=0x7f131400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314007cc0 con 0x7f13240fccf0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131400f040 con 0x7f13240fccf0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1324117d60 con 0x7f13240fccf0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131400c8f0 con 0x7f13240fccf0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.887+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1324118220 con 0x7f13240fccf0 2023-10-07T23:15:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.888+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1324108560 con 0x7f13240fccf0 2023-10-07T23:15:49.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.890+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f131400cd20 con 0x7f13240fccf0 2023-10-07T23:15:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.891+0000 7f1312ffd700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130c09b770 0x7f130c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.891+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f13140c4c60 con 0x7f13240fccf0 2023-10-07T23:15:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.892+0000 7f1321d9b700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130c09b770 0x7f130c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.892+0000 7f1321d9b700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130c09b770 0x7f130c09dc20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1318004890 tx=0x7f1318009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:49.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:49.895+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f131404da00 con 0x7f13240fccf0 2023-10-07T23:15:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:50.147+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1324103bc0 con 0x7f13240fccf0 2023-10-07T23:15:50.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:49 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3057887261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:50.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:49 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3214144761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:50.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:49 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3057887261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:50.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:49 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3214144761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:50.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:49 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3057887261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:50.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:49 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3214144761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:50.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:50.997+0000 7f1312ffd700 1 -- 172.21.15.67:0/1041866061 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1314019020 con 0x7f13240fccf0 2023-10-07T23:15:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130c09b770 msgr2=0x7f130c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130c09b770 0x7f130c09dc20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1318004890 tx=0x7f1318009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:51.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 msgr2=0x7f1324071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 0x7f1324071930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f131400ebf0 tx=0x7f131400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 shutdown_connections 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130c09b770 0x7f130c09dc20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13240fe890 0x7f13240704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13240fccf0 0x7f1324071930 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 --2- 172.21.15.67:0/1041866061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13240fdf00 0x7f132406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 >> 172.21.15.67:0/1041866061 conn(0x7f13240f8450 msgr2=0x7f13240fa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 shutdown_connections 2023-10-07T23:15:51.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.001+0000 7f132359e700 1 -- 172.21.15.67:0/1041866061 wait complete. 2023-10-07T23:15:51.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-07T23:15:51.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/1523526315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a380ffe90 msgr2=0x7f0a381022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/1523526315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a380ffe90 0x7f0a381022f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a280049c0 tx=0x7f0a28009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/1523526315 shutdown_connections 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/1523526315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38102980 0x7f0a38104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/1523526315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a38066d10 0x7f0a380ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/1523526315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a380ffe90 0x7f0a381022f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/1523526315 >> 172.21.15.67:0/1523526315 conn(0x7f0a380f7370 msgr2=0x7f0a380f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:51.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/1523526315 shutdown_connections 2023-10-07T23:15:51.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.225+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/1523526315 wait complete. 2023-10-07T23:15:51.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 Processor -- start 2023-10-07T23:15:51.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 -- start start 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38066d10 0x7f0a3810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a380ffe90 0x7f0a3810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38102980 0x7f0a38115020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38115630 con 0x7f0a38066d10 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381157a0 con 0x7f0a38102980 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3ed9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38115960 con 0x7f0a380ffe90 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38066d10 0x7f0a3810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38066d10 0x7f0a3810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52862/0 (socket says 172.21.15.67:52862) 2023-10-07T23:15:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3e59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38102980 0x7f0a38115020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3dd9b700 1 -- 172.21.15.67:0/3402805427 learned_addr learned my addr 172.21.15.67:0/3402805427 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.226+0000 7f0a3d59a700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a380ffe90 0x7f0a3810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3e59c700 1 -- 172.21.15.67:0/3402805427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a380ffe90 msgr2=0x7f0a3810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3e59c700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a380ffe90 0x7f0a3810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3e59c700 1 -- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38066d10 msgr2=0x7f0a3810da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3e59c700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38066d10 0x7f0a3810da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3e59c700 1 -- 172.21.15.67:0/3402805427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28005040 con 0x7f0a38102980 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3d59a700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a380ffe90 0x7f0a3810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3e59c700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38102980 0x7f0a38115020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000ece0 tx=0x7f0a3000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300049b0 con 0x7f0a38102980 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3000f040 con 0x7f0a38102980 2023-10-07T23:15:51.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a38115be0 con 0x7f0a38102980 2023-10-07T23:15:51.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300094a0 con 0x7f0a38102980 2023-10-07T23:15:51.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.227+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381160d0 con 0x7f0a38102980 2023-10-07T23:15:51.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.228+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a380f8ea0 con 0x7f0a38102980 2023-10-07T23:15:51.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.230+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f0a30007920 con 0x7f0a38102980 2023-10-07T23:15:51.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.232+0000 7f0a2e7fc700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a2409b770 0x7f0a2409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:51.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.232+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f0a300c5d40 con 0x7f0a38102980 2023-10-07T23:15:51.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.232+0000 7f0a3dd9b700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a2409b770 0x7f0a2409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:51.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.233+0000 7f0a3dd9b700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a2409b770 0x7f0a2409dc20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004bc0 tx=0x7f0a3400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:51.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.233+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f0a30091310 con 0x7f0a38102980 2023-10-07T23:15:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:50 smithi067 ceph-mon[82723]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:50 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1041866061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-07T23:15:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:50 smithi123 ceph-mon[122926]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:50 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1041866061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-07T23:15:51.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:50 smithi086 ceph-mon[122131]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:51.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:50 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1041866061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-07T23:15:51.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:51.482+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f0a38108700 con 0x7f0a38102980 2023-10-07T23:15:52.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.004+0000 7f0a2e7fc700 1 -- 172.21.15.67:0/3402805427 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0a38108700 con 0x7f0a38102980 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a2409b770 msgr2=0x7f0a2409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a2409b770 0x7f0a2409dc20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004bc0 tx=0x7f0a3400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38102980 msgr2=0x7f0a38115020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38102980 0x7f0a38115020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000ece0 tx=0x7f0a3000b040 comp rx=0 tx=0).stop 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 shutdown_connections 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a38102980 0x7f0a38115020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0a2409b770 0x7f0a2409dc20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a38066d10 0x7f0a3810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 --2- 172.21.15.67:0/3402805427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a380ffe90 0x7f0a3810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 >> 172.21.15.67:0/3402805427 conn(0x7f0a380f7370 msgr2=0x7f0a380fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:52.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 shutdown_connections 2023-10-07T23:15:52.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.007+0000 7f0a3ed9d700 1 -- 172.21.15.67:0/3402805427 wait complete. 2023-10-07T23:15:52.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-07T23:15:52.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 -- 172.21.15.67:0/2363470930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 msgr2=0x7f4258102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:52.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 --2- 172.21.15.67:0/2363470930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258102550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42480049c0 tx=0x7f4248009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:52.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 -- 172.21.15.67:0/2363470930 shutdown_connections 2023-10-07T23:15:52.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 --2- 172.21.15.67:0/2363470930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 0x7f4258104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 --2- 172.21.15.67:0/2363470930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 0x7f42580ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 --2- 172.21.15.67:0/2363470930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258102550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 -- 172.21.15.67:0/2363470930 >> 172.21.15.67:0/2363470930 conn(0x7f42580f7370 msgr2=0x7f42580f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 -- 172.21.15.67:0/2363470930 shutdown_connections 2023-10-07T23:15:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.222+0000 7f425f39d700 1 -- 172.21.15.67:0/2363470930 wait complete. 2023-10-07T23:15:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 Processor -- start 2023-10-07T23:15:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 -- start start 2023-10-07T23:15:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 0x7f425810fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 0x7f4258117210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258117860 con 0x7f4258100170 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42581179d0 con 0x7f4258102a90 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.223+0000 7f425f39d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258117b90 con 0x7f42580fd850 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425e39b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 0x7f425810fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425e39b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 0x7f425810fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34972/0 (socket says 172.21.15.67:34972) 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425e39b700 1 -- 172.21.15.67:0/487833114 learned_addr learned my addr 172.21.15.67:0/487833114 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425eb9c700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 0x7f4258117210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 -- 172.21.15.67:0/487833114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 msgr2=0x7f425810fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 0x7f425810fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 -- 172.21.15.67:0/487833114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 msgr2=0x7f4258117210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 0x7f4258117210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 -- 172.21.15.67:0/487833114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248005040 con 0x7f4258100170 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425eb9c700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 0x7f4258117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f425db9a700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258110190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42480049c0 tx=0x7f4248009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.224+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424800e070 con 0x7f4258100170 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.225+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4248004160 con 0x7f4258100170 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.225+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258117e10 con 0x7f4258100170 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.225+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424801cea0 con 0x7f4258100170 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.225+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42581182e0 con 0x7f4258100170 2023-10-07T23:15:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.226+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42581045c0 con 0x7f4258100170 2023-10-07T23:15:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.228+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f424801c580 con 0x7f4258100170 2023-10-07T23:15:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.230+0000 7f424effd700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f424409b7f0 0x7f424409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.230+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f42480c6320 con 0x7f4258100170 2023-10-07T23:15:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.230+0000 7f425e39b700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f424409b7f0 0x7f424409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.230+0000 7f425e39b700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f424409b7f0 0x7f424409dca0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4254008fd0 tx=0x7f425400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.234+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f42480918f0 con 0x7f4258100170 2023-10-07T23:15:52.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:52 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1041866061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-07T23:15:52.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:52 smithi067 ceph-mon[82723]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:52.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:52 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3402805427' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-07T23:15:52.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:52 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-07T23:15:52.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:52 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1041866061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-07T23:15:52.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:52 smithi123 ceph-mon[122926]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:52.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:52 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3402805427' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-07T23:15:52.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:52 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-07T23:15:52.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:52 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1041866061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-07T23:15:52.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:52 smithi086 ceph-mon[122131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:52.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:52 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3402805427' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-07T23:15:52.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:52 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-07T23:15:52.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:52.480+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f4258108540 con 0x7f4258100170 2023-10-07T23:15:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.009+0000 7f424effd700 1 -- 172.21.15.67:0/487833114 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4248094f10 con 0x7f4258100170 2023-10-07T23:15:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f424409b7f0 msgr2=0x7f424409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f424409b7f0 0x7f424409dca0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4254008fd0 tx=0x7f425400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 msgr2=0x7f4258110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258110190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42480049c0 tx=0x7f4248009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 shutdown_connections 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4258102a90 0x7f4258117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f424409b7f0 0x7f424409dca0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42580fd850 0x7f425810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 --2- 172.21.15.67:0/487833114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4258100170 0x7f4258110190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 >> 172.21.15.67:0/487833114 conn(0x7f42580f7370 msgr2=0x7f42580fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.012+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 shutdown_connections 2023-10-07T23:15:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.013+0000 7f425f39d700 1 -- 172.21.15.67:0/487833114 wait complete. 2023-10-07T23:15:53.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 -- 172.21.15.67:0/1137802656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fccf0 msgr2=0x7fb0200fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 --2- 172.21.15.67:0/1137802656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fccf0 0x7fb0200fd100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb0100049c0 tx=0x7fb010009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 -- 172.21.15.67:0/1137802656 shutdown_connections 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 --2- 172.21.15.67:0/1137802656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0200fe890 0x7fb020105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 --2- 172.21.15.67:0/1137802656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fccf0 0x7fb0200fd100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 --2- 172.21.15.67:0/1137802656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0200fdf00 0x7fb0200fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.227+0000 7fb02579d700 1 -- 172.21.15.67:0/1137802656 >> 172.21.15.67:0/1137802656 conn(0x7fb0200f8450 msgr2=0x7fb0200fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.228+0000 7fb02579d700 1 -- 172.21.15.67:0/1137802656 shutdown_connections 2023-10-07T23:15:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.228+0000 7fb02579d700 1 -- 172.21.15.67:0/1137802656 wait complete. 2023-10-07T23:15:53.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.228+0000 7fb02579d700 1 Processor -- start 2023-10-07T23:15:53.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 -- start start 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0200fccf0 0x7fb02010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 0x7fb02010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0200fe890 0x7fb020114f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb020115550 con 0x7fb0200fdf00 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201156c0 con 0x7fb0200fccf0 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb02579d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb020115880 con 0x7fb0200fe890 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 0x7fb02010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 0x7fb02010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55982/0 (socket says 172.21.15.67:55982) 2023-10-07T23:15:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 -- 172.21.15.67:0/1558298401 learned_addr learned my addr 172.21.15.67:0/1558298401 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01ffff700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0200fccf0 0x7fb02010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb024f9c700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0200fe890 0x7fb020114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 -- 172.21.15.67:0/1558298401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0200fe890 msgr2=0x7fb020114f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0200fe890 0x7fb020114f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 -- 172.21.15.67:0/1558298401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0200fccf0 msgr2=0x7fb02010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0200fccf0 0x7fb02010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.229+0000 7fb01f7fe700 1 -- 172.21.15.67:0/1558298401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010005040 con 0x7fb0200fdf00 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.230+0000 7fb01f7fe700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 0x7fb02010deb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb01400ebc0 tx=0x7fb01400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.230+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014007cc0 con 0x7fb0200fdf00 2023-10-07T23:15:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.230+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01400f040 con 0x7fb0200fdf00 2023-10-07T23:15:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.230+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb020115b00 con 0x7fb0200fdf00 2023-10-07T23:15:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.230+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140094a0 con 0x7fb0200fdf00 2023-10-07T23:15:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.230+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb020115ff0 con 0x7fb0200fdf00 2023-10-07T23:15:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.231+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb020103bb0 con 0x7fb0200fdf00 2023-10-07T23:15:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.233+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb01400c600 con 0x7fb0200fdf00 2023-10-07T23:15:53.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.234+0000 7fb01cff9700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb00809b770 0x7fb00809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:53.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.235+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fb0140c4d60 con 0x7fb0200fdf00 2023-10-07T23:15:53.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.235+0000 7fb01ffff700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb00809b770 0x7fb00809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.235+0000 7fb01ffff700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb00809b770 0x7fb00809dc20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0100098c0 tx=0x7fb010009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:53.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.238+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fb01404da00 con 0x7fb0200fdf00 2023-10-07T23:15:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-07T23:15:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-07T23:15:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-07T23:15:53.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-07T23:15:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-07T23:15:53.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:53 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/487833114' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-07T23:15:53.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-07T23:15:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:53 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/487833114' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-07T23:15:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-07T23:15:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-07T23:15:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-07T23:15:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-07T23:15:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-07T23:15:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-07T23:15:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-07T23:15:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:53 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/487833114' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-07T23:15:53.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.489+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fb02002b310 con 0x7fb0200fdf00 2023-10-07T23:15:53.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.834+0000 7fb01cff9700 1 -- 172.21.15.67:0/1558298401 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb014019020 con 0x7fb0200fdf00 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb00809b770 msgr2=0x7fb00809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb00809b770 0x7fb00809dc20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0100098c0 tx=0x7fb010009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 msgr2=0x7fb02010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 0x7fb02010deb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb01400ebc0 tx=0x7fb01400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 shutdown_connections 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0200fccf0 0x7fb02010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb00809b770 0x7fb00809dc20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0200fe890 0x7fb020114f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 --2- 172.21.15.67:0/1558298401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0200fdf00 0x7fb02010deb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:53.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 >> 172.21.15.67:0/1558298401 conn(0x7fb0200f8450 msgr2=0x7fb0200fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:53.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 shutdown_connections 2023-10-07T23:15:53.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:53.836+0000 7fb02579d700 1 -- 172.21.15.67:0/1558298401 wait complete. 2023-10-07T23:15:53.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:15:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 -- 172.21.15.67:0/304553601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4070570 msgr2=0x7fd5b40709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/304553601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4070570 0x7fd5b40709c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80049c0 tx=0x7fd5a8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 -- 172.21.15.67:0/304553601 shutdown_connections 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/304553601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/304553601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4104b80 0x7fd5b4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/304553601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4070570 0x7fd5b40709c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 -- 172.21.15.67:0/304553601 >> 172.21.15.67:0/304553601 conn(0x7fd5b40f8460 msgr2=0x7fd5b40fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 -- 172.21.15.67:0/304553601 shutdown_connections 2023-10-07T23:15:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.053+0000 7fd5b89fe700 1 -- 172.21.15.67:0/304553601 wait complete. 2023-10-07T23:15:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 Processor -- start 2023-10-07T23:15:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 -- start start 2023-10-07T23:15:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4070570 0x7fd5b410fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4104b80 0x7fd5b41172a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b41178e0 con 0x7fd5b4071bc0 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4117a50 con 0x7fd5b4104b80 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b89fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4117c10 con 0x7fd5b4070570 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4104b80 0x7fd5b41172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55996/0 (socket says 172.21.15.67:55996) 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b259c700 1 -- 172.21.15.67:0/3264229572 learned_addr learned my addr 172.21.15.67:0/3264229572 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.054+0000 7fd5b2d9d700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4070570 0x7fd5b410fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b259c700 1 -- 172.21.15.67:0/3264229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4070570 msgr2=0x7fd5b410fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b259c700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4070570 0x7fd5b410fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b259c700 1 -- 172.21.15.67:0/3264229572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4104b80 msgr2=0x7fd5b41172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b259c700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4104b80 0x7fd5b41172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b259c700 1 -- 172.21.15.67:0/3264229572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8005040 con 0x7fd5b4071bc0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b359e700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4104b80 0x7fd5b41172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b259c700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4110250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009180 tx=0x7fd5a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8020070 con 0x7fd5b4071bc0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5a8009520 con 0x7fd5b4071bc0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b4117e90 con 0x7fd5b4071bc0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800e070 con 0x7fd5b4071bc0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.055+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b4118230 con 0x7fd5b4071bc0 2023-10-07T23:15:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.056+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b410a740 con 0x7fd5b4071bc0 2023-10-07T23:15:54.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.058+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd5a8009690 con 0x7fd5b4071bc0 2023-10-07T23:15:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.059+0000 7fd59b7fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd59c09b770 0x7fd59c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.059+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fd5a80c6270 con 0x7fd5b4071bc0 2023-10-07T23:15:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.060+0000 7fd5b2d9d700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd59c09b770 0x7fd59c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.060+0000 7fd5b2d9d700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd59c09b770 0x7fd59c09dc20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4111200 tx=0x7fd5a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:54.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.064+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd5a80918a0 con 0x7fd5b4071bc0 2023-10-07T23:15:54.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] up:active 2023-10-07T23:15:54.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/487833114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-07T23:15:54.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-07T23:15:54.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1558298401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] up:active 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1558298401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: daemon mds.d assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:creating} 3 up:standby-replay 5 up:standby 2023-10-07T23:15:54.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:54 smithi067 ceph-mon[82723]: daemon mds.d is now active in filesystem cephfs as rank 3 2023-10-07T23:15:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.315+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5b41032f0 con 0x7fd5b4071bc0 2023-10-07T23:15:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.316+0000 7fd59b7fe700 1 -- 172.21.15.67:0/3264229572 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10747 (secure 0 0 0) 0x7fd5a8091030 con 0x7fd5b4071bc0 2023-10-07T23:15:54.316 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:54.317 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:53.835081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:creating","state_seq":1,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:15:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.318+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd59c09b770 msgr2=0x7fd59c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.318+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd59c09b770 0x7fd59c09dc20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4111200 tx=0x7fd5a400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.318+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 msgr2=0x7fd5b4110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.318+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4110250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009180 tx=0x7fd5a8009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.318+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 shutdown_connections 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b4104b80 0x7fd5b41172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd59c09b770 0x7fd59c09dc20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4070570 0x7fd5b410fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 --2- 172.21.15.67:0/3264229572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5b4071bc0 0x7fd5b4110250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 >> 172.21.15.67:0/3264229572 conn(0x7fd5b40f8460 msgr2=0x7fd5b40fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 shutdown_connections 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.319+0000 7fd5b89fe700 1 -- 172.21.15.67:0/3264229572 wait complete. 2023-10-07T23:15:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 24 2023-10-07T23:15:54.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:15:54.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] up:active 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/487833114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1558298401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] up:active 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1558298401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: daemon mds.d assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-07T23:15:54.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:creating} 3 up:standby-replay 5 up:standby 2023-10-07T23:15:54.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:54 smithi123 ceph-mon[122926]: daemon mds.d is now active in filesystem cephfs as rank 3 2023-10-07T23:15:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] up:active 2023-10-07T23:15:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/487833114' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-07T23:15:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1558298401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] up:active 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1558298401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: daemon mds.d assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:creating} 3 up:standby-replay 5 up:standby 2023-10-07T23:15:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:54 smithi086 ceph-mon[122131]: daemon mds.d is now active in filesystem cephfs as rank 3 2023-10-07T23:15:54.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 -- 172.21.15.67:0/3139011154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c074200 msgr2=0x7f527c074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 --2- 172.21.15.67:0/3139011154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c074200 0x7f527c074a90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5274009d60 tx=0x7f5274004d10 comp rx=0 tx=0).stop 2023-10-07T23:15:54.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 -- 172.21.15.67:0/3139011154 shutdown_connections 2023-10-07T23:15:54.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 --2- 172.21.15.67:0/3139011154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 --2- 172.21.15.67:0/3139011154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c074200 0x7f527c074a90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 --2- 172.21.15.67:0/3139011154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c073850 0x7f527c073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 -- 172.21.15.67:0/3139011154 >> 172.21.15.67:0/3139011154 conn(0x7f527c0fa630 msgr2=0x7f527c0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:54.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 -- 172.21.15.67:0/3139011154 shutdown_connections 2023-10-07T23:15:54.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.526+0000 7f528219d700 1 -- 172.21.15.67:0/3139011154 wait complete. 2023-10-07T23:15:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.527+0000 7f528219d700 1 Processor -- start 2023-10-07T23:15:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.527+0000 7f528219d700 1 -- start start 2023-10-07T23:15:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.527+0000 7f528219d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.527+0000 7f528219d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c073850 0x7f527c114370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528219d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c074200 0x7f527c11b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528219d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c11ba10 con 0x7f527c073850 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528219d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c11bb80 con 0x7f527c0725f0 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528219d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c11bd40 con 0x7f527c074200 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528119b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c073850 0x7f527c114370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528119b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:43034/0 (socket says 172.21.15.67:43034) 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528119b700 1 -- 172.21.15.67:0/268772824 learned_addr learned my addr 172.21.15.67:0/268772824 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528199c700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c074200 0x7f527c11b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 -- 172.21.15.67:0/268772824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c074200 msgr2=0x7f527c11b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c074200 0x7f527c11b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 -- 172.21.15.67:0/268772824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 msgr2=0x7f527c113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c113e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 -- 172.21.15.67:0/268772824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274009940 con 0x7f527c073850 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528119b700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.528+0000 7f528099a700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c073850 0x7f527c114370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f526c009fd0 tx=0x7f526c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.529+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c004e70 con 0x7f527c073850 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.529+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526c00f040 con 0x7f527c073850 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.529+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00c820 con 0x7f527c073850 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.529+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c11bfc0 con 0x7f527c073850 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.529+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c11c450 con 0x7f527c073850 2023-10-07T23:15:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.530+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c10b990 con 0x7f527c073850 2023-10-07T23:15:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.532+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f526c00cc50 con 0x7f527c073850 2023-10-07T23:15:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.534+0000 7f5271ffb700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f526809b7f0 0x7f526809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.534+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f526c0c4d70 con 0x7f527c073850 2023-10-07T23:15:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.534+0000 7f528119b700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f526809b7f0 0x7f526809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.535+0000 7f528119b700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f526809b7f0 0x7f526809dca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52780041d0 tx=0x7f527800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.537+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f526c090340 con 0x7f527c073850 2023-10-07T23:15:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.742+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f527c02b450 con 0x7f527c073850 2023-10-07T23:15:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.743+0000 7f5271ffb700 1 -- 172.21.15.67:0/268772824 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f526c019020 con 0x7f527c073850 2023-10-07T23:15:54.744 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:54.744 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:15:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f526809b7f0 msgr2=0x7f526809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f526809b7f0 0x7f526809dca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52780041d0 tx=0x7f527800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c073850 msgr2=0x7f527c114370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c073850 0x7f527c114370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f526c009fd0 tx=0x7f526c00b040 comp rx=0 tx=0).stop 2023-10-07T23:15:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 shutdown_connections 2023-10-07T23:15:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f527c0725f0 0x7f527c113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f526809b7f0 0x7f526809dca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c074200 0x7f527c11b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 --2- 172.21.15.67:0/268772824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c073850 0x7f527c114370 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 >> 172.21.15.67:0/268772824 conn(0x7f527c0fa630 msgr2=0x7f527c0fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.745+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 shutdown_connections 2023-10-07T23:15:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.746+0000 7f528219d700 1 -- 172.21.15.67:0/268772824 wait complete. 2023-10-07T23:15:54.764 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T23:15:54.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:15:54.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.959+0000 7fd88559d700 1 -- 172.21.15.67:0/1432917787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8800fefe0 msgr2=0x7fd8800ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.959+0000 7fd88559d700 1 --2- 172.21.15.67:0/1432917787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8800fefe0 0x7fd8800ff3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8700049c0 tx=0x7fd870009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 -- 172.21.15.67:0/1432917787 shutdown_connections 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 --2- 172.21.15.67:0/1432917787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd880100b80 0x7fd880107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 --2- 172.21.15.67:0/1432917787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8800fefe0 0x7fd8800ff3f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 --2- 172.21.15.67:0/1432917787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8801001f0 0x7fd880100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 -- 172.21.15.67:0/1432917787 >> 172.21.15.67:0/1432917787 conn(0x7fd8800fa790 msgr2=0x7fd8800fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 -- 172.21.15.67:0/1432917787 shutdown_connections 2023-10-07T23:15:54.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 -- 172.21.15.67:0/1432917787 wait complete. 2023-10-07T23:15:54.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.960+0000 7fd88559d700 1 Processor -- start 2023-10-07T23:15:54.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 -- start start 2023-10-07T23:15:54.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800fefe0 0x7fd88010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 0x7fd880110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd880100b80 0x7fd8801171b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801177f0 con 0x7fd880100b80 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880117960 con 0x7fd8801001f0 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd88559d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880117b20 con 0x7fd8800fefe0 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 0x7fd880110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 0x7fd880110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:43056/0 (socket says 172.21.15.67:43056) 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd87f7fe700 1 -- 172.21.15.67:0/1795063847 learned_addr learned my addr 172.21.15.67:0/1795063847 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd884d9c700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd880100b80 0x7fd8801171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.961+0000 7fd87ffff700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800fefe0 0x7fd88010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87f7fe700 1 -- 172.21.15.67:0/1795063847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800fefe0 msgr2=0x7fd88010fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87f7fe700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800fefe0 0x7fd88010fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87f7fe700 1 -- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd880100b80 msgr2=0x7fd8801171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:54.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87f7fe700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd880100b80 0x7fd8801171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87f7fe700 1 -- 172.21.15.67:0/1795063847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870005040 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87f7fe700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 0x7fd880110160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd87400ebf0 tx=0x7fd87400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874007cc0 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880117da0 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd87400f040 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874009460 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.962+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880118250 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.963+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880105e70 con 0x7fd8801001f0 2023-10-07T23:15:54.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.965+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd87400c680 con 0x7fd8801001f0 2023-10-07T23:15:54.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.967+0000 7fd87cff9700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd86809b770 0x7fd86809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:54.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.967+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fd8740c5160 con 0x7fd8801001f0 2023-10-07T23:15:54.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.967+0000 7fd87ffff700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd86809b770 0x7fd86809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:54.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.968+0000 7fd87ffff700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd86809b770 0x7fd86809dc20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd870004890 tx=0x7fd870009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:54.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:54.969+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd874090730 con 0x7fd8801001f0 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3264229572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/268772824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] up:active 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: Cluster is now healthy 2023-10-07T23:15:55.012 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:active} 3 up:standby-replay 5 up:standby 2023-10-07T23:15:55.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-10-07T23:15:55.013 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:55 smithi067 ceph-mon[82723]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-10-07T23:15:55.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.172+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd88002b500 con 0x7fd86809b770 2023-10-07T23:15:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.188+0000 7fd87cff9700 1 -- 172.21.15.67:0/1795063847 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423010 (secure 0 0 0) 0x7fd88002b500 con 0x7fd86809b770 2023-10-07T23:15:55.194 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:55.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd86809b770 msgr2=0x7fd86809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd86809b770 0x7fd86809dc20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd870004890 tx=0x7fd870009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 msgr2=0x7fd880110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 0x7fd880110160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd87400ebf0 tx=0x7fd87400b040 comp rx=0 tx=0).stop 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 shutdown_connections 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd86809b770 0x7fd86809dc20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd880100b80 0x7fd8801171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800fefe0 0x7fd88010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 --2- 172.21.15.67:0/1795063847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801001f0 0x7fd880110160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 >> 172.21.15.67:0/1795063847 conn(0x7fd8800fa790 msgr2=0x7fd8800fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 shutdown_connections 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.193+0000 7fd88559d700 1 -- 172.21.15.67:0/1795063847 wait complete. 2023-10-07T23:15:55.198 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:15:55.213 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-10-07T23:15:53.926544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596385,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":286,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3580792,"kb_used_data":13336,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121361032,"statfs":{"total":1151940427776,"available":1148273696768,"internally_reserved":0,"allocated":13656064,"data_stored":7491265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3711,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003420"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365769+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:15:31.365769+0000","last_peered":"2023-10-07T23:15:31.365769+0000","last_clean":"2023-10-07T23:15:31.365769+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:15:31.365769+0000","last_undegraded":"2023-10-07T23:15:31.365769+0000","last_fullsized":"2023-10-07T23:15:31.365769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.379445+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:15:31.379445+0000","last_peered":"2023-10-07T23:15:31.379445+0000","last_clean":"2023-10-07T23:15:31.379445+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:15:31.379445+0000","last_undegraded":"2023-10-07T23:15:31.379445+0000","last_fullsized":"2023-10-07T23:15:31.379445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.377008+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:15:31.377008+0000","last_peered":"2023-10-07T23:15:31.377008+0000","last_clean":"2023-10-07T23:15:31.377008+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:15:31.377008+0000","last_undegraded":"2023-10-07T23:15:31.377008+0000","last_fullsized":"2023-10-07T23:15:31.377008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.366980+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:15:31.366980+0000","last_peered":"2023-10-07T23:15:31.366980+0000","last_clean":"2023-10-07T23:15:31.366980+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:15:31.366980+0000","last_undegraded":"2023-10-07T23:15:31.366980+0000","last_fullsized":"2023-10-07T23:15:31.366980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370125+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:15:31.370125+0000","last_peered":"2023-10-07T23:15:31.370125+0000","last_clean":"2023-10-07T23:15:31.370125+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:15:31.370125+0000","last_undegraded":"2023-10-07T23:15:31.370125+0000","last_fullsized":"2023-10-07T23:15:31.370125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620266+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:15:31.620266+0000","last_peered":"2023-10-07T23:15:31.620266+0000","last_clean":"2023-10-07T23:15:31.620266+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:15:31.620266+0000","last_undegraded":"2023-10-07T23:15:31.620266+0000","last_fullsized":"2023-10-07T23:15:31.620266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371682+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:15:31.371682+0000","last_peered":"2023-10-07T23:15:31.371682+0000","last_clean":"2023-10-07T23:15:31.371682+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:15:31.371682+0000","last_undegraded":"2023-10-07T23:15:31.371682+0000","last_fullsized":"2023-10-07T23:15:31.371682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372800+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:15:31.372800+0000","last_peered":"2023-10-07T23:15:31.372800+0000","last_clean":"2023-10-07T23:15:31.372800+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:15:31.372800+0000","last_undegraded":"2023-10-07T23:15:31.372800+0000","last_fullsized":"2023-10-07T23:15:31.372800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372605+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:15:31.372605+0000","last_peered":"2023-10-07T23:15:31.372605+0000","last_clean":"2023-10-07T23:15:31.372605+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:15:31.372605+0000","last_undegraded":"2023-10-07T23:15:31.372605+0000","last_fullsized":"2023-10-07T23:15:31.372605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620876+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:15:31.620876+0000","last_peered":"2023-10-07T23:15:31.620876+0000","last_clean":"2023-10-07T23:15:31.620876+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:15:31.620876+0000","last_undegraded":"2023-10-07T23:15:31.620876+0000","last_fullsized":"2023-10-07T23:15:31.620876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372347+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:15:31.372347+0000","last_peered":"2023-10-07T23:15:31.372347+0000","last_clean":"2023-10-07T23:15:31.372347+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:15:31.372347+0000","last_undegraded":"2023-10-07T23:15:31.372347+0000","last_fullsized":"2023-10-07T23:15:31.372347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.366879+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:15:31.366879+0000","last_peered":"2023-10-07T23:15:31.366879+0000","last_clean":"2023-10-07T23:15:31.366879+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:15:31.366879+0000","last_undegraded":"2023-10-07T23:15:31.366879+0000","last_fullsized":"2023-10-07T23:15:31.366879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369784+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:15:31.369784+0000","last_peered":"2023-10-07T23:15:31.369784+0000","last_clean":"2023-10-07T23:15:31.369784+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:15:31.369784+0000","last_undegraded":"2023-10-07T23:15:31.369784+0000","last_fullsized":"2023-10-07T23:15:31.369784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376755+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:15:31.376755+0000","last_peered":"2023-10-07T23:15:31.376755+0000","last_clean":"2023-10-07T23:15:31.376755+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:15:31.376755+0000","last_undegraded":"2023-10-07T23:15:31.376755+0000","last_fullsized":"2023-10-07T23:15:31.376755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.383512+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:15:31.383512+0000","last_peered":"2023-10-07T23:15:31.383512+0000","last_clean":"2023-10-07T23:15:31.383512+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:15:31.383512+0000","last_undegraded":"2023-10-07T23:15:31.383512+0000","last_fullsized":"2023-10-07T23:15:31.383512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371531+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:15:31.371531+0000","last_peered":"2023-10-07T23:15:31.371531+0000","last_clean":"2023-10-07T23:15:31.371531+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:15:31.371531+0000","last_undegraded":"2023-10-07T23:15:31.371531+0000","last_fullsized":"2023-10-07T23:15:31.371531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365916+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:15:31.365916+0000","last_peered":"2023-10-07T23:15:31.365916+0000","last_clean":"2023-10-07T23:15:31.365916+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:15:31.365916+0000","last_undegraded":"2023-10-07T23:15:31.365916+0000","last_fullsized":"2023-10-07T23:15:31.365916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.377279+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:15:31.377279+0000","last_peered":"2023-10-07T23:15:31.377279+0000","last_clean":"2023-10-07T23:15:31.377279+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:15:31.377279+0000","last_undegraded":"2023-10-07T23:15:31.377279+0000","last_fullsized":"2023-10-07T23:15:31.377279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374813+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:15:31.374813+0000","last_peered":"2023-10-07T23:15:31.374813+0000","last_clean":"2023-10-07T23:15:31.374813+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:15:31.374813+0000","last_undegraded":"2023-10-07T23:15:31.374813+0000","last_fullsized":"2023-10-07T23:15:31.374813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.389673+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:15:31.389673+0000","last_peered":"2023-10-07T23:15:31.389673+0000","last_clean":"2023-10-07T23:15:31.389673+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:15:31.389673+0000","last_undegraded":"2023-10-07T23:15:31.389673+0000","last_fullsized":"2023-10-07T23:15:31.389673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.023841+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:15:53.023841+0000","last_peered":"2023-10-07T23:15:53.023841+0000","last_clean":"2023-10-07T23:15:53.023841+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:15:53.023841+0000","last_undegraded":"2023-10-07T23:15:53.023841+0000","last_fullsized":"2023-10-07T23:15:53.023841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373919+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:15:31.373919+0000","last_peered":"2023-10-07T23:15:31.373919+0000","last_clean":"2023-10-07T23:15:31.373919+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:15:31.373919+0000","last_undegraded":"2023-10-07T23:15:31.373919+0000","last_fullsized":"2023-10-07T23:15:31.373919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386744+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:15:31.386744+0000","last_peered":"2023-10-07T23:15:31.386744+0000","last_clean":"2023-10-07T23:15:31.386744+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:15:31.386744+0000","last_undegraded":"2023-10-07T23:15:31.386744+0000","last_fullsized":"2023-10-07T23:15:31.386744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374006+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:15:31.374006+0000","last_peered":"2023-10-07T23:15:31.374006+0000","last_clean":"2023-10-07T23:15:31.374006+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:15:31.374006+0000","last_undegraded":"2023-10-07T23:15:31.374006+0000","last_fullsized":"2023-10-07T23:15:31.374006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380432+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:15:31.380432+0000","last_peered":"2023-10-07T23:15:31.380432+0000","last_clean":"2023-10-07T23:15:31.380432+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:15:31.380432+0000","last_undegraded":"2023-10-07T23:15:31.380432+0000","last_fullsized":"2023-10-07T23:15:31.380432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.389586+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:15:31.389586+0000","last_peered":"2023-10-07T23:15:31.389586+0000","last_clean":"2023-10-07T23:15:31.389586+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:15:31.389586+0000","last_undegraded":"2023-10-07T23:15:31.389586+0000","last_fullsized":"2023-10-07T23:15:31.389586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.020240+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:15:53.020240+0000","last_peered":"2023-10-07T23:15:53.020240+0000","last_clean":"2023-10-07T23:15:53.020240+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:15:53.020240+0000","last_undegraded":"2023-10-07T23:15:53.020240+0000","last_fullsized":"2023-10-07T23:15:53.020240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372782+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:15:31.372782+0000","last_peered":"2023-10-07T23:15:31.372782+0000","last_clean":"2023-10-07T23:15:31.372782+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:15:31.372782+0000","last_undegraded":"2023-10-07T23:15:31.372782+0000","last_fullsized":"2023-10-07T23:15:31.372782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386725+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:15:31.386725+0000","last_peered":"2023-10-07T23:15:31.386725+0000","last_clean":"2023-10-07T23:15:31.386725+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:15:31.386725+0000","last_undegraded":"2023-10-07T23:15:31.386725+0000","last_fullsized":"2023-10-07T23:15:31.386725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370386+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:15:31.370386+0000","last_peered":"2023-10-07T23:15:31.370386+0000","last_clean":"2023-10-07T23:15:31.370386+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:15:31.370386+0000","last_undegraded":"2023-10-07T23:15:31.370386+0000","last_fullsized":"2023-10-07T23:15:31.370386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376424+0000","last_change":"2023-10-07T23:15:27.197818+0000","last_active":"2023-10-07T23:15:31.376424+0000","last_peered":"2023-10-07T23:15:31.376424+0000","last_clean":"2023-10-07T23:15:31.376424+0000","last_became_active":"2023-10-07T23:15:27.197522+0000","last_became_peered":"2023-10-07T23:15:27.197522+0000","last_unstale":"2023-10-07T23:15:31.376424+0000","last_undegraded":"2023-10-07T23:15:31.376424+0000","last_fullsized":"2023-10-07T23:15:31.376424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:11.128855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380564+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:15:31.380564+0000","last_peered":"2023-10-07T23:15:31.380564+0000","last_clean":"2023-10-07T23:15:31.380564+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:15:31.380564+0000","last_undegraded":"2023-10-07T23:15:31.380564+0000","last_fullsized":"2023-10-07T23:15:31.380564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.017175+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:15:52.017175+0000","last_peered":"2023-10-07T23:15:52.017175+0000","last_clean":"2023-10-07T23:15:52.017175+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:15:52.017175+0000","last_undegraded":"2023-10-07T23:15:52.017175+0000","last_fullsized":"2023-10-07T23:15:52.017175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370472+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:15:31.370472+0000","last_peered":"2023-10-07T23:15:31.370472+0000","last_clean":"2023-10-07T23:15:31.370472+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:15:31.370472+0000","last_undegraded":"2023-10-07T23:15:31.370472+0000","last_fullsized":"2023-10-07T23:15:31.370472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.387209+0000","last_change":"2023-10-07T23:15:31.386751+0000","last_active":"2023-10-07T23:15:31.387209+0000","last_peered":"2023-10-07T23:15:31.387209+0000","last_clean":"2023-10-07T23:15:31.387209+0000","last_became_active":"2023-10-07T23:15:31.386370+0000","last_became_peered":"2023-10-07T23:15:31.386370+0000","last_unstale":"2023-10-07T23:15:31.387209+0000","last_undegraded":"2023-10-07T23:15:31.387209+0000","last_fullsized":"2023-10-07T23:15:31.387209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:26:43.668892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620283+0000","last_change":"2023-10-07T23:15:26.349502+0000","last_active":"2023-10-07T23:15:31.620283+0000","last_peered":"2023-10-07T23:15:31.620283+0000","last_clean":"2023-10-07T23:15:31.620283+0000","last_became_active":"2023-10-07T23:15:26.349182+0000","last_became_peered":"2023-10-07T23:15:26.349182+0000","last_unstale":"2023-10-07T23:15:31.620283+0000","last_undegraded":"2023-10-07T23:15:31.620283+0000","last_fullsized":"2023-10-07T23:15:31.620283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:40.762251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.375571+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:15:31.375571+0000","last_peered":"2023-10-07T23:15:31.375571+0000","last_clean":"2023-10-07T23:15:31.375571+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:15:31.375571+0000","last_undegraded":"2023-10-07T23:15:31.375571+0000","last_fullsized":"2023-10-07T23:15:31.375571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.393275+0000","last_change":"2023-10-07T23:15:31.392966+0000","last_active":"2023-10-07T23:15:31.393275+0000","last_peered":"2023-10-07T23:15:31.393275+0000","last_clean":"2023-10-07T23:15:31.393275+0000","last_became_active":"2023-10-07T23:15:31.392612+0000","last_became_peered":"2023-10-07T23:15:31.392612+0000","last_unstale":"2023-10-07T23:15:31.393275+0000","last_undegraded":"2023-10-07T23:15:31.393275+0000","last_fullsized":"2023-10-07T23:15:31.393275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:09:36.871330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.023716+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:15:53.023716+0000","last_peered":"2023-10-07T23:15:53.023716+0000","last_clean":"2023-10-07T23:15:53.023716+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:15:53.023716+0000","last_undegraded":"2023-10-07T23:15:53.023716+0000","last_fullsized":"2023-10-07T23:15:53.023716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372903+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:15:31.372903+0000","last_peered":"2023-10-07T23:15:31.372903+0000","last_clean":"2023-10-07T23:15:31.372903+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:15:31.372903+0000","last_undegraded":"2023-10-07T23:15:31.372903+0000","last_fullsized":"2023-10-07T23:15:31.372903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.395981+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:15:31.395981+0000","last_peered":"2023-10-07T23:15:31.395981+0000","last_clean":"2023-10-07T23:15:31.395981+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:15:31.395981+0000","last_undegraded":"2023-10-07T23:15:31.395981+0000","last_fullsized":"2023-10-07T23:15:31.395981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.348021+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:15:34.348021+0000","last_peered":"2023-10-07T23:15:34.348021+0000","last_clean":"2023-10-07T23:15:34.348021+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:15:34.348021+0000","last_undegraded":"2023-10-07T23:15:34.348021+0000","last_fullsized":"2023-10-07T23:15:34.348021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376554+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:15:31.376554+0000","last_peered":"2023-10-07T23:15:31.376554+0000","last_clean":"2023-10-07T23:15:31.376554+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:15:31.376554+0000","last_undegraded":"2023-10-07T23:15:31.376554+0000","last_fullsized":"2023-10-07T23:15:31.376554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.385921+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:15:31.385921+0000","last_peered":"2023-10-07T23:15:31.385921+0000","last_clean":"2023-10-07T23:15:31.385921+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:15:31.385921+0000","last_undegraded":"2023-10-07T23:15:31.385921+0000","last_fullsized":"2023-10-07T23:15:31.385921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.015458+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:15:52.015458+0000","last_peered":"2023-10-07T23:15:52.015458+0000","last_clean":"2023-10-07T23:15:52.015458+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:15:52.015458+0000","last_undegraded":"2023-10-07T23:15:52.015458+0000","last_fullsized":"2023-10-07T23:15:52.015458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372294+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:15:31.372294+0000","last_peered":"2023-10-07T23:15:31.372294+0000","last_clean":"2023-10-07T23:15:31.372294+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:15:31.372294+0000","last_undegraded":"2023-10-07T23:15:31.372294+0000","last_fullsized":"2023-10-07T23:15:31.372294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.394523+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:15:31.394523+0000","last_peered":"2023-10-07T23:15:31.394523+0000","last_clean":"2023-10-07T23:15:31.394523+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:15:31.394523+0000","last_undegraded":"2023-10-07T23:15:31.394523+0000","last_fullsized":"2023-10-07T23:15:31.394523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016581+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:15:52.016581+0000","last_peered":"2023-10-07T23:15:52.016581+0000","last_clean":"2023-10-07T23:15:52.016581+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:15:52.016581+0000","last_undegraded":"2023-10-07T23:15:52.016581+0000","last_fullsized":"2023-10-07T23:15:52.016581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380519+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:15:31.380519+0000","last_peered":"2023-10-07T23:15:31.380519+0000","last_clean":"2023-10-07T23:15:31.380519+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:15:31.380519+0000","last_undegraded":"2023-10-07T23:15:31.380519+0000","last_fullsized":"2023-10-07T23:15:31.380519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386111+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:15:31.386111+0000","last_peered":"2023-10-07T23:15:31.386111+0000","last_clean":"2023-10-07T23:15:31.386111+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:15:31.386111+0000","last_undegraded":"2023-10-07T23:15:31.386111+0000","last_fullsized":"2023-10-07T23:15:31.386111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.347613+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:15:34.347613+0000","last_peered":"2023-10-07T23:15:34.347613+0000","last_clean":"2023-10-07T23:15:34.347613+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:15:34.347613+0000","last_undegraded":"2023-10-07T23:15:34.347613+0000","last_fullsized":"2023-10-07T23:15:34.347613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373002+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:15:31.373002+0000","last_peered":"2023-10-07T23:15:31.373002+0000","last_clean":"2023-10-07T23:15:31.373002+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:15:31.373002+0000","last_undegraded":"2023-10-07T23:15:31.373002+0000","last_fullsized":"2023-10-07T23:15:31.373002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.391512+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:15:31.391512+0000","last_peered":"2023-10-07T23:15:31.391512+0000","last_clean":"2023-10-07T23:15:31.391512+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:15:31.391512+0000","last_undegraded":"2023-10-07T23:15:31.391512+0000","last_fullsized":"2023-10-07T23:15:31.391512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016432+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:15:52.016432+0000","last_peered":"2023-10-07T23:15:52.016432+0000","last_clean":"2023-10-07T23:15:52.016432+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:15:52.016432+0000","last_undegraded":"2023-10-07T23:15:52.016432+0000","last_fullsized":"2023-10-07T23:15:52.016432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370864+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:15:31.370864+0000","last_peered":"2023-10-07T23:15:31.370864+0000","last_clean":"2023-10-07T23:15:31.370864+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:15:31.370864+0000","last_undegraded":"2023-10-07T23:15:31.370864+0000","last_fullsized":"2023-10-07T23:15:31.370864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386772+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:15:31.386772+0000","last_peered":"2023-10-07T23:15:31.386772+0000","last_clean":"2023-10-07T23:15:31.386772+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:15:31.386772+0000","last_undegraded":"2023-10-07T23:15:31.386772+0000","last_fullsized":"2023-10-07T23:15:31.386772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374111+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:15:31.374111+0000","last_peered":"2023-10-07T23:15:31.374111+0000","last_clean":"2023-10-07T23:15:31.374111+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:15:31.374111+0000","last_undegraded":"2023-10-07T23:15:31.374111+0000","last_fullsized":"2023-10-07T23:15:31.374111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.368794+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:15:31.368794+0000","last_peered":"2023-10-07T23:15:31.368794+0000","last_clean":"2023-10-07T23:15:31.368794+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:15:31.368794+0000","last_undegraded":"2023-10-07T23:15:31.368794+0000","last_fullsized":"2023-10-07T23:15:31.368794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.385305+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:15:31.385305+0000","last_peered":"2023-10-07T23:15:31.385305+0000","last_clean":"2023-10-07T23:15:31.385305+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:15:31.385305+0000","last_undegraded":"2023-10-07T23:15:31.385305+0000","last_fullsized":"2023-10-07T23:15:31.385305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.017231+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:15:52.017231+0000","last_peered":"2023-10-07T23:15:52.017231+0000","last_clean":"2023-10-07T23:15:52.017231+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:15:52.017231+0000","last_undegraded":"2023-10-07T23:15:52.017231+0000","last_fullsized":"2023-10-07T23:15:52.017231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372062+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:15:31.372062+0000","last_peered":"2023-10-07T23:15:31.372062+0000","last_clean":"2023-10-07T23:15:31.372062+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:15:31.372062+0000","last_undegraded":"2023-10-07T23:15:31.372062+0000","last_fullsized":"2023-10-07T23:15:31.372062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.398606+0000","last_change":"2023-10-07T23:15:31.397451+0000","last_active":"2023-10-07T23:15:31.398606+0000","last_peered":"2023-10-07T23:15:31.398606+0000","last_clean":"2023-10-07T23:15:31.398606+0000","last_became_active":"2023-10-07T23:15:31.397001+0000","last_became_peered":"2023-10-07T23:15:31.397001+0000","last_unstale":"2023-10-07T23:15:31.398606+0000","last_undegraded":"2023-10-07T23:15:31.398606+0000","last_fullsized":"2023-10-07T23:15:31.398606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:35:45.208477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365152+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:15:31.365152+0000","last_peered":"2023-10-07T23:15:31.365152+0000","last_clean":"2023-10-07T23:15:31.365152+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:15:31.365152+0000","last_undegraded":"2023-10-07T23:15:31.365152+0000","last_fullsized":"2023-10-07T23:15:31.365152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371467+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:15:31.371467+0000","last_peered":"2023-10-07T23:15:31.371467+0000","last_clean":"2023-10-07T23:15:31.371467+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:15:31.371467+0000","last_undegraded":"2023-10-07T23:15:31.371467+0000","last_fullsized":"2023-10-07T23:15:31.371467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.393511+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:15:31.393511+0000","last_peered":"2023-10-07T23:15:31.393511+0000","last_clean":"2023-10-07T23:15:31.393511+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:15:31.393511+0000","last_undegraded":"2023-10-07T23:15:31.393511+0000","last_fullsized":"2023-10-07T23:15:31.393511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.022068+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:15:53.022068+0000","last_peered":"2023-10-07T23:15:53.022068+0000","last_clean":"2023-10-07T23:15:53.022068+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:15:53.022068+0000","last_undegraded":"2023-10-07T23:15:53.022068+0000","last_fullsized":"2023-10-07T23:15:53.022068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376151+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:15:31.376151+0000","last_peered":"2023-10-07T23:15:31.376151+0000","last_clean":"2023-10-07T23:15:31.376151+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:15:31.376151+0000","last_undegraded":"2023-10-07T23:15:31.376151+0000","last_fullsized":"2023-10-07T23:15:31.376151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.389661+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:15:31.389661+0000","last_peered":"2023-10-07T23:15:31.389661+0000","last_clean":"2023-10-07T23:15:31.389661+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:15:31.389661+0000","last_undegraded":"2023-10-07T23:15:31.389661+0000","last_fullsized":"2023-10-07T23:15:31.389661+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370594+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:15:31.370594+0000","last_peered":"2023-10-07T23:15:31.370594+0000","last_clean":"2023-10-07T23:15:31.370594+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:15:31.370594+0000","last_undegraded":"2023-10-07T23:15:31.370594+0000","last_fullsized":"2023-10-07T23:15:31.370594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365124+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:15:31.365124+0000","last_peered":"2023-10-07T23:15:31.365124+0000","last_clean":"2023-10-07T23:15:31.365124+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:15:31.365124+0000","last_undegraded":"2023-10-07T23:15:31.365124+0000","last_fullsized":"2023-10-07T23:15:31.365124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.622769+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:15:31.622769+0000","last_peered":"2023-10-07T23:15:31.622769+0000","last_clean":"2023-10-07T23:15:31.622769+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:15:31.622769+0000","last_undegraded":"2023-10-07T23:15:31.622769+0000","last_fullsized":"2023-10-07T23:15:31.622769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376822+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:15:31.376822+0000","last_peered":"2023-10-07T23:15:31.376822+0000","last_clean":"2023-10-07T23:15:31.376822+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:15:31.376822+0000","last_undegraded":"2023-10-07T23:15:31.376822+0000","last_fullsized":"2023-10-07T23:15:31.376822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620719+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:15:31.620719+0000","last_peered":"2023-10-07T23:15:31.620719+0000","last_clean":"2023-10-07T23:15:31.620719+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:15:31.620719+0000","last_undegraded":"2023-10-07T23:15:31.620719+0000","last_fullsized":"2023-10-07T23:15:31.620719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.396051+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:15:31.396051+0000","last_peered":"2023-10-07T23:15:31.396051+0000","last_clean":"2023-10-07T23:15:31.396051+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:15:31.396051+0000","last_undegraded":"2023-10-07T23:15:31.396051+0000","last_fullsized":"2023-10-07T23:15:31.396051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372517+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:15:31.372517+0000","last_peered":"2023-10-07T23:15:31.372517+0000","last_clean":"2023-10-07T23:15:31.372517+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:15:31.372517+0000","last_undegraded":"2023-10-07T23:15:31.372517+0000","last_fullsized":"2023-10-07T23:15:31.372517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620703+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:15:31.620703+0000","last_peered":"2023-10-07T23:15:31.620703+0000","last_clean":"2023-10-07T23:15:31.620703+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:15:31.620703+0000","last_undegraded":"2023-10-07T23:15:31.620703+0000","last_fullsized":"2023-10-07T23:15:31.620703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623366+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:15:31.623366+0000","last_peered":"2023-10-07T23:15:31.623366+0000","last_clean":"2023-10-07T23:15:31.623366+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:15:31.623366+0000","last_undegraded":"2023-10-07T23:15:31.623366+0000","last_fullsized":"2023-10-07T23:15:31.623366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376698+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:15:31.376698+0000","last_peered":"2023-10-07T23:15:31.376698+0000","last_clean":"2023-10-07T23:15:31.376698+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:15:31.376698+0000","last_undegraded":"2023-10-07T23:15:31.376698+0000","last_fullsized":"2023-10-07T23:15:31.376698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.022743+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:15:53.022743+0000","last_peered":"2023-10-07T23:15:53.022743+0000","last_clean":"2023-10-07T23:15:53.022743+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:15:53.022743+0000","last_undegraded":"2023-10-07T23:15:53.022743+0000","last_fullsized":"2023-10-07T23:15:53.022743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386149+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:15:31.386149+0000","last_peered":"2023-10-07T23:15:31.386149+0000","last_clean":"2023-10-07T23:15:31.386149+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:15:31.386149+0000","last_undegraded":"2023-10-07T23:15:31.386149+0000","last_fullsized":"2023-10-07T23:15:31.386149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.375648+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:15:31.375648+0000","last_peered":"2023-10-07T23:15:31.375648+0000","last_clean":"2023-10-07T23:15:31.375648+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:15:31.375648+0000","last_undegraded":"2023-10-07T23:15:31.375648+0000","last_fullsized":"2023-10-07T23:15:31.375648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016958+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:15:52.016958+0000","last_peered":"2023-10-07T23:15:52.016958+0000","last_clean":"2023-10-07T23:15:52.016958+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:15:52.016958+0000","last_undegraded":"2023-10-07T23:15:52.016958+0000","last_fullsized":"2023-10-07T23:15:52.016958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370819+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:15:31.370819+0000","last_peered":"2023-10-07T23:15:31.370819+0000","last_clean":"2023-10-07T23:15:31.370819+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:15:31.370819+0000","last_undegraded":"2023-10-07T23:15:31.370819+0000","last_fullsized":"2023-10-07T23:15:31.370819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623248+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:15:31.623248+0000","last_peered":"2023-10-07T23:15:31.623248+0000","last_clean":"2023-10-07T23:15:31.623248+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:15:31.623248+0000","last_undegraded":"2023-10-07T23:15:31.623248+0000","last_fullsized":"2023-10-07T23:15:31.623248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.367855+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:15:31.367855+0000","last_peered":"2023-10-07T23:15:31.367855+0000","last_clean":"2023-10-07T23:15:31.367855+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:15:31.367855+0000","last_undegraded":"2023-10-07T23:15:31.367855+0000","last_fullsized":"2023-10-07T23:15:31.367855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.214102+0000","last_change":"2023-10-07T23:15:26.352454+0000","last_active":"2023-10-07T23:15:34.214102+0000","last_peered":"2023-10-07T23:15:34.214102+0000","last_clean":"2023-10-07T23:15:34.214102+0000","last_became_active":"2023-10-07T23:15:26.352156+0000","last_became_peered":"2023-10-07T23:15:26.352156+0000","last_unstale":"2023-10-07T23:15:34.214102+0000","last_undegraded":"2023-10-07T23:15:34.214102+0000","last_fullsized":"2023-10-07T23:15:34.214102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:21.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380182+0000","last_change":"2023-10-07T23:15:31.379007+0000","last_active":"2023-10-07T23:15:31.380182+0000","last_peered":"2023-10-07T23:15:31.380182+0000","last_clean":"2023-10-07T23:15:31.380182+0000","last_became_active":"2023-10-07T23:15:31.378549+0000","last_became_peered":"2023-10-07T23:15:31.378549+0000","last_unstale":"2023-10-07T23:15:31.380182+0000","last_undegraded":"2023-10-07T23:15:31.380182+0000","last_fullsized":"2023-10-07T23:15:31.380182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:47.745473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.348255+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:15:34.348255+0000","last_peered":"2023-10-07T23:15:34.348255+0000","last_clean":"2023-10-07T23:15:34.348255+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:15:34.348255+0000","last_undegraded":"2023-10-07T23:15:34.348255+0000","last_fullsized":"2023-10-07T23:15:34.348255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.368777+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:15:31.368777+0000","last_peered":"2023-10-07T23:15:31.368777+0000","last_clean":"2023-10-07T23:15:31.368777+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:15:31.368777+0000","last_undegraded":"2023-10-07T23:15:31.368777+0000","last_fullsized":"2023-10-07T23:15:31.368777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623563+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:15:31.623563+0000","last_peered":"2023-10-07T23:15:31.623563+0000","last_clean":"2023-10-07T23:15:31.623563+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:15:31.623563+0000","last_undegraded":"2023-10-07T23:15:31.623563+0000","last_fullsized":"2023-10-07T23:15:31.623563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.023306+0000","last_change":"2023-10-07T23:15:26.363682+0000","last_active":"2023-10-07T23:15:53.023306+0000","last_peered":"2023-10-07T23:15:53.023306+0000","last_clean":"2023-10-07T23:15:53.023306+0000","last_became_active":"2023-10-07T23:15:26.362955+0000","last_became_peered":"2023-10-07T23:15:26.362955+0000","last_unstale":"2023-10-07T23:15:53.023306+0000","last_undegraded":"2023-10-07T23:15:53.023306+0000","last_fullsized":"2023-10-07T23:15:53.023306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:41.150623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371255+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:15:31.371255+0000","last_peered":"2023-10-07T23:15:31.371255+0000","last_clean":"2023-10-07T23:15:31.371255+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:15:31.371255+0000","last_undegraded":"2023-10-07T23:15:31.371255+0000","last_fullsized":"2023-10-07T23:15:31.371255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380428+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:15:31.380428+0000","last_peered":"2023-10-07T23:15:31.380428+0000","last_clean":"2023-10-07T23:15:31.380428+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:15:31.380428+0000","last_undegraded":"2023-10-07T23:15:31.380428+0000","last_fullsized":"2023-10-07T23:15:31.380428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016888+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:15:52.016888+0000","last_peered":"2023-10-07T23:15:52.016888+0000","last_clean":"2023-10-07T23:15:52.016888+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:15:52.016888+0000","last_undegraded":"2023-10-07T23:15:52.016888+0000","last_fullsized":"2023-10-07T23:15:52.016888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.379537+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:15:31.379537+0000","last_peered":"2023-10-07T23:15:31.379537+0000","last_clean":"2023-10-07T23:15:31.379537+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:15:31.379537+0000","last_undegraded":"2023-10-07T23:15:31.379537+0000","last_fullsized":"2023-10-07T23:15:31.379537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623340+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:15:31.623340+0000","last_peered":"2023-10-07T23:15:31.623340+0000","last_clean":"2023-10-07T23:15:31.623340+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:15:31.623340+0000","last_undegraded":"2023-10-07T23:15:31.623340+0000","last_fullsized":"2023-10-07T23:15:31.623340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373136+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:15:31.373136+0000","last_peered":"2023-10-07T23:15:31.373136+0000","last_clean":"2023-10-07T23:15:31.373136+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:15:31.373136+0000","last_undegraded":"2023-10-07T23:15:31.373136+0000","last_fullsized":"2023-10-07T23:15:31.373136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370060+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:15:31.370060+0000","last_peered":"2023-10-07T23:15:31.370060+0000","last_clean":"2023-10-07T23:15:31.370060+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:15:31.370060+0000","last_undegraded":"2023-10-07T23:15:31.370060+0000","last_fullsized":"2023-10-07T23:15:31.370060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.397249+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:15:31.397249+0000","last_peered":"2023-10-07T23:15:31.397249+0000","last_clean":"2023-10-07T23:15:31.397249+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:15:31.397249+0000","last_undegraded":"2023-10-07T23:15:31.397249+0000","last_fullsized":"2023-10-07T23:15:31.397249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.017471+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:15:52.017471+0000","last_peered":"2023-10-07T23:15:52.017471+0000","last_clean":"2023-10-07T23:15:52.017471+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:15:52.017471+0000","last_undegraded":"2023-10-07T23:15:52.017471+0000","last_fullsized":"2023-10-07T23:15:52.017471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.364899+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:15:31.364899+0000","last_peered":"2023-10-07T23:15:31.364899+0000","last_clean":"2023-10-07T23:15:31.364899+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:15:31.364899+0000","last_undegraded":"2023-10-07T23:15:31.364899+0000","last_fullsized":"2023-10-07T23:15:31.364899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.385895+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:15:31.385895+0000","last_peered":"2023-10-07T23:15:31.385895+0000","last_clean":"2023-10-07T23:15:31.385895+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:15:31.385895+0000","last_undegraded":"2023-10-07T23:15:31.385895+0000","last_fullsized":"2023-10-07T23:15:31.385895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373249+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:15:31.373249+0000","last_peered":"2023-10-07T23:15:31.373249+0000","last_clean":"2023-10-07T23:15:31.373249+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:15:31.373249+0000","last_undegraded":"2023-10-07T23:15:31.373249+0000","last_fullsized":"2023-10-07T23:15:31.373249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.378191+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:15:31.378191+0000","last_peered":"2023-10-07T23:15:31.378191+0000","last_clean":"2023-10-07T23:15:31.378191+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:15:31.378191+0000","last_undegraded":"2023-10-07T23:15:31.378191+0000","last_fullsized":"2023-10-07T23:15:31.378191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623298+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:15:31.623298+0000","last_peered":"2023-10-07T23:15:31.623298+0000","last_clean":"2023-10-07T23:15:31.623298+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:15:31.623298+0000","last_undegraded":"2023-10-07T23:15:31.623298+0000","last_fullsized":"2023-10-07T23:15:31.623298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016419+0000","last_change":"2023-10-07T23:15:26.357820+0000","last_active":"2023-10-07T23:15:52.016419+0000","last_peered":"2023-10-07T23:15:52.016419+0000","last_clean":"2023-10-07T23:15:52.016419+0000","last_became_active":"2023-10-07T23:15:26.357545+0000","last_became_peered":"2023-10-07T23:15:26.357545+0000","last_unstale":"2023-10-07T23:15:52.016419+0000","last_undegraded":"2023-10-07T23:15:52.016419+0000","last_fullsized":"2023-10-07T23:15:52.016419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:25:15.446891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372269+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:15:31.372269+0000","last_peered":"2023-10-07T23:15:31.372269+0000","last_clean":"2023-10-07T23:15:31.372269+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:15:31.372269+0000","last_undegraded":"2023-10-07T23:15:31.372269+0000","last_fullsized":"2023-10-07T23:15:31.372269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380899+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:15:31.380899+0000","last_peered":"2023-10-07T23:15:31.380899+0000","last_clean":"2023-10-07T23:15:31.380899+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:15:31.380899+0000","last_undegraded":"2023-10-07T23:15:31.380899+0000","last_fullsized":"2023-10-07T23:15:31.380899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376215+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:15:31.376215+0000","last_peered":"2023-10-07T23:15:31.376215+0000","last_clean":"2023-10-07T23:15:31.376215+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:15:31.376215+0000","last_undegraded":"2023-10-07T23:15:31.376215+0000","last_fullsized":"2023-10-07T23:15:31.376215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372429+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:15:31.372429+0000","last_peered":"2023-10-07T23:15:31.372429+0000","last_clean":"2023-10-07T23:15:31.372429+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:15:31.372429+0000","last_undegraded":"2023-10-07T23:15:31.372429+0000","last_fullsized":"2023-10-07T23:15:31.372429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.396751+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:15:31.396751+0000","last_peered":"2023-10-07T23:15:31.396751+0000","last_clean":"2023-10-07T23:15:31.396751+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:15:31.396751+0000","last_undegraded":"2023-10-07T23:15:31.396751+0000","last_fullsized":"2023-10-07T23:15:31.396751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376104+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:15:31.376104+0000","last_peered":"2023-10-07T23:15:31.376104+0000","last_clean":"2023-10-07T23:15:31.376104+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:15:31.376104+0000","last_undegraded":"2023-10-07T23:15:31.376104+0000","last_fullsized":"2023-10-07T23:15:31.376104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371831+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:15:31.371831+0000","last_peered":"2023-10-07T23:15:31.371831+0000","last_clean":"2023-10-07T23:15:31.371831+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:15:31.371831+0000","last_undegraded":"2023-10-07T23:15:31.371831+0000","last_fullsized":"2023-10-07T23:15:31.371831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380633+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:15:31.380633+0000","last_peered":"2023-10-07T23:15:31.380633+0000","last_clean":"2023-10-07T23:15:31.380633+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:15:31.380633+0000","last_undegraded":"2023-10-07T23:15:31.380633+0000","last_fullsized":"2023-10-07T23:15:31.380633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369993+0000","last_change":"2023-10-07T23:15:26.359589+0000","last_active":"2023-10-07T23:15:31.369993+0000","last_peered":"2023-10-07T23:15:31.369993+0000","last_clean":"2023-10-07T23:15:31.369993+0000","last_became_active":"2023-10-07T23:15:26.359222+0000","last_became_peered":"2023-10-07T23:15:26.359222+0000","last_unstale":"2023-10-07T23:15:31.369993+0000","last_undegraded":"2023-10-07T23:15:31.369993+0000","last_fullsized":"2023-10-07T23:15:31.369993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:09.393841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372074+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:15:31.372074+0000","last_peered":"2023-10-07T23:15:31.372074+0000","last_clean":"2023-10-07T23:15:31.372074+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:15:31.372074+0000","last_undegraded":"2023-10-07T23:15:31.372074+0000","last_fullsized":"2023-10-07T23:15:31.372074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.364826+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:15:31.364826+0000","last_peered":"2023-10-07T23:15:31.364826+0000","last_clean":"2023-10-07T23:15:31.364826+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:15:31.364826+0000","last_undegraded":"2023-10-07T23:15:31.364826+0000","last_fullsized":"2023-10-07T23:15:31.364826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374782+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:15:31.374782+0000","last_peered":"2023-10-07T23:15:31.374782+0000","last_clean":"2023-10-07T23:15:31.374782+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:15:31.374782+0000","last_undegraded":"2023-10-07T23:15:31.374782+0000","last_fullsized":"2023-10-07T23:15:31.374782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373134+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:15:31.373134+0000","last_peered":"2023-10-07T23:15:31.373134+0000","last_clean":"2023-10-07T23:15:31.373134+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:15:31.373134+0000","last_undegraded":"2023-10-07T23:15:31.373134+0000","last_fullsized":"2023-10-07T23:15:31.373134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376784+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:15:31.376784+0000","last_peered":"2023-10-07T23:15:31.376784+0000","last_clean":"2023-10-07T23:15:31.376784+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:15:31.376784+0000","last_undegraded":"2023-10-07T23:15:31.376784+0000","last_fullsized":"2023-10-07T23:15:31.376784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369038+0000","last_change":"2023-10-07T23:15:27.206430+0000","last_active":"2023-10-07T23:15:31.369038+0000","last_peered":"2023-10-07T23:15:31.369038+0000","last_clean":"2023-10-07T23:15:31.369038+0000","last_became_active":"2023-10-07T23:15:27.206044+0000","last_became_peered":"2023-10-07T23:15:27.206044+0000","last_unstale":"2023-10-07T23:15:31.369038+0000","last_undegraded":"2023-10-07T23:15:31.369038+0000","last_fullsized":"2023-10-07T23:15:31.369038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:57.365683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365107+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:15:31.365107+0000","last_peered":"2023-10-07T23:15:31.365107+0000","last_clean":"2023-10-07T23:15:31.365107+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:15:31.365107+0000","last_undegraded":"2023-10-07T23:15:31.365107+0000","last_fullsized":"2023-10-07T23:15:31.365107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373419+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:15:31.373419+0000","last_peered":"2023-10-07T23:15:31.373419+0000","last_clean":"2023-10-07T23:15:31.373419+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:15:31.373419+0000","last_undegraded":"2023-10-07T23:15:31.373419+0000","last_fullsized":"2023-10-07T23:15:31.373419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371337+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:15:31.371337+0000","last_peered":"2023-10-07T23:15:31.371337+0000","last_clean":"2023-10-07T23:15:31.371337+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:15:31.371337+0000","last_undegraded":"2023-10-07T23:15:31.371337+0000","last_fullsized":"2023-10-07T23:15:31.371337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376665+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:15:31.376665+0000","last_peered":"2023-10-07T23:15:31.376665+0000","last_clean":"2023-10-07T23:15:31.376665+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:15:31.376665+0000","last_undegraded":"2023-10-07T23:15:31.376665+0000","last_fullsized":"2023-10-07T23:15:31.376665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.375387+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:15:31.375387+0000","last_peered":"2023-10-07T23:15:31.375387+0000","last_clean":"2023-10-07T23:15:31.375387+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:15:31.375387+0000","last_undegraded":"2023-10-07T23:15:31.375387+0000","last_fullsized":"2023-10-07T23:15:31.375387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372592+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:15:31.372592+0000","last_peered":"2023-10-07T23:15:31.372592+0000","last_clean":"2023-10-07T23:15:31.372592+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:15:31.372592+0000","last_undegraded":"2023-10-07T23:15:31.372592+0000","last_fullsized":"2023-10-07T23:15:31.372592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369687+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:15:31.369687+0000","last_peered":"2023-10-07T23:15:31.369687+0000","last_clean":"2023-10-07T23:15:31.369687+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:15:31.369687+0000","last_undegraded":"2023-10-07T23:15:31.369687+0000","last_fullsized":"2023-10-07T23:15:31.369687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365104+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:15:31.365104+0000","last_peered":"2023-10-07T23:15:31.365104+0000","last_clean":"2023-10-07T23:15:31.365104+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:15:31.365104+0000","last_undegraded":"2023-10-07T23:15:31.365104+0000","last_fullsized":"2023-10-07T23:15:31.365104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6017,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":17107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383695,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446908,"statfs":{"total":95995035648,"available":95689633792,"internally_reserved":0,"allocated":978944,"data_stored":475491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.51200000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.79400000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.75900000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298856,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446296,"statfs":{"total":95995035648,"available":95689007104,"internally_reserved":0,"allocated":1605632,"data_stored":1070179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35599999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.504}]},{"osd":5,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56000000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.57899999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.73799999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298260,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446892,"statfs":{"total":95995035648,"available":95689617408,"internally_reserved":0,"allocated":995328,"data_stored":476247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68200000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.88,"5min":1.054,"15min":1.054},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.627}]},{"osd":2,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.84699999999999998,"15min":0.86099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.325,"15min":1.325},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":1.278,"15min":1.278},"last":0.79900000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.67600000000000005,"15min":0.69799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76700000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":1.385,"15min":1.385},"last":0.60899999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.52800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97799999999999998,"5min":1.345,"15min":1.345},"last":0.752},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.84799999999999998,"15min":0.873},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.82299999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.90100000000000002,"15min":0.92700000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97599999999999998,"5min":1.579,"15min":1.579},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.96399999999999997,"15min":0.99299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0649999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.113,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.018,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2310000000000001,"5min":1.603,"15min":1.603},"last":0.74099999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.026,"15min":1.056},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.038,"5min":1.782,"15min":1.782},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.96599999999999997,"15min":0.998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0529999999999999,"5min":1.956,"15min":1.956},"last":0.78600000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298256,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446896,"statfs":{"total":95995035648,"available":95689621504,"internally_reserved":0,"allocated":991232,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.755,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76300000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.66200000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.72799999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63900000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.498},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75600000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56299999999999994}]},{"osd":4,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.746}]},{"osd":5,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82699999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84299999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96799999999999997,"5min":1.597,"15min":1.597},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.872,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88400000000000001,"5min":1,"15min":1},"last":0.82599999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.59899999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92900000000000005,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96399999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Sat Oct 7 23:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.91600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298828,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446324,"statfs":{"total":95995035648,"available":95689035776,"internally_reserved":0,"allocated":1576960,"data_stored":1066413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.502},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.44600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44600000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69099999999999995,"15min":0.68000000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54000000000000004}]},{"osd":8,"last update":"Sat Oct 7 23:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98599999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68000000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":475168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70999999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70999999999999996,"5min":0.878,"15min":0.878},"last":0.64700000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.70999999999999996,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.5700000000000003,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.56299999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68899999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.877,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.95599999999999996,"5min":1.014,"15min":1.014},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75,"15min":0.751},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.099,"15min":1.099},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.127,"5min":0.87,"15min":0.82699999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.84599999999999997}]},{"osd":8,"last update":"Sat Oct 7 23:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85799999999999998,"15min":0.85599999999999998},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.0509999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88800000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0580000000000001,"5min":1.165,"15min":1.165},"last":0.81000000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:15:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":6.5890000000000004,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446900,"statfs":{"total":95995035648,"available":95689625600,"internally_reserved":0,"allocated":987136,"data_stored":478883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.624},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78700000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92800000000000005,"5min":1.014,"15min":1.014},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.42599999999999999},"min":{"1min":0.28499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.501}]},{"osd":5,"last update":"Sat Oct 7 23:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.56599999999999995,"15min":0.53400000000000003},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72599999999999998}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.752}]},{"osd":10,"last update":"Sat Oct 7 23:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446900,"statfs":{"total":95995035648,"available":95689625600,"internally_reserved":0,"allocated":987136,"data_stored":475834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.874},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":1.022,"15min":1.022},"last":0.79200000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.45500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69699999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.80800000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89100000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.52100000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":1.155,"15min":1.155},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.83199999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78900000000000003,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95499999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93700000000000006,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.91900000000000004}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2150000000000001,"5min":1.323,"15min":1.323},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.97299999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.159},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.99299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298256,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446896,"statfs":{"total":95995035648,"available":95689621504,"internally_reserved":0,"allocated":991232,"data_stored":478917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52900000000000003,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.747,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.502},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83399999999999996,"5min":0.871,"15min":0.871},"last":0.80200000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.752}]},{"osd":5,"last update":"Sat Oct 7 23:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.439,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65100000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.46200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76500000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.436,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76800000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68400000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81999999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84699999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.85799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446900,"statfs":{"total":95995035648,"available":95689625600,"internally_reserved":0,"allocated":987136,"data_stored":475595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.32300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.436},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.875,"5min":0.999,"15min":0.999},"last":0.60999999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.59899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.0109999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.75},{"interface":"front","average":{"1min":0.625,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.60599999999999998},"min":{"1min":0.53400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.65300000000000002,"15min":0.60899999999999999},"min":{"1min":0.57699999999999996,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":1.218,"15min":1.218},"last":0.84699999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.80400000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68700000000000006,"15min":0.67400000000000004},"min":{"1min":0.498,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Oct 7 23:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.374,"15min":0.374},"max":{"1min":1.048,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.94999999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.71399999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2230000000000001,"5min":1.288,"15min":1.288},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0029999999999999,"15min":1.0189999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.169,"5min":1.51,"15min":1.51},"last":0.97299999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]}]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446904,"statfs":{"total":95995035648,"available":95689629696,"internally_reserved":0,"allocated":983040,"data_stored":475513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.36499999999999999},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.372},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.38800000000000001,"15min":0.35299999999999998},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.49399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.49399999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.46999999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.53800000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.51800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.55500000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.63700000000000001},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.73299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77300000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78000000000000003}]},{"osd":8,"last update":"Sat Oct 7 23:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.80200000000000005,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63500000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298848,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446304,"statfs":{"total":95995035648,"available":95689015296,"internally_reserved":0,"allocated":1597440,"data_stored":1066936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.499,"15min":0.47599999999999998},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.502,"15min":0.47999999999999998},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46300000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.38300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.505,"15min":0.48899999999999999},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.374,"15min":0.35499999999999998},"min":{"1min":0.25,"5min":0.217,"15min":0.217},"max":{"1min":0.59399999999999997,"5min":0.629,"15min":0.629},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.371},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.438}]},{"osd":3,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Sat Oct 7 23:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80100000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Oct 7 23:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78000000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:15:55.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:15:55.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3264229572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/268772824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] up:active 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: Cluster is now healthy 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:active} 3 up:standby-replay 5 up:standby 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-10-07T23:15:55.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:55 smithi123 ceph-mon[122926]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-10-07T23:15:55.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2342491541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fcce0 msgr2=0x7f9a5c0fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2342491541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fcce0 0x7f9a5c0fd0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0049c0 tx=0x7f9a4c009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:55.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2342491541 shutdown_connections 2023-10-07T23:15:55.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2342491541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0fe880 0x7f9a5c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2342491541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fcce0 0x7f9a5c0fd0f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2342491541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c0fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2342491541 >> 172.21.15.67:0/2342491541 conn(0x7f9a5c0f8460 msgr2=0x7f9a5c0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2342491541 shutdown_connections 2023-10-07T23:15:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.445+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2342491541 wait complete. 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3264229572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/268772824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] up:active 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: Cluster is now healthy 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:active} 3 up:standby-replay 5 up:standby 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-10-07T23:15:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:55 smithi086 ceph-mon[122131]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 Processor -- start 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 -- start start 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0fcce0 0x7f9a5c10fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5c0fe880 0x7f9a5c117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c1177d0 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c117940 con 0x7f9a5c0fcce0 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a5b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c117b00 con 0x7f9a5c0fe880 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a59d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a59d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56058/0 (socket says 172.21.15.67:56058) 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.446+0000 7f9a59d9b700 1 -- 172.21.15.67:0/2716108002 learned_addr learned my addr 172.21.15.67:0/2716108002 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a5a59c700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0fcce0 0x7f9a5c10fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a5ad9d700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5c0fe880 0x7f9a5c117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a59d9b700 1 -- 172.21.15.67:0/2716108002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5c0fe880 msgr2=0x7f9a5c117190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a59d9b700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5c0fe880 0x7f9a5c117190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a59d9b700 1 -- 172.21.15.67:0/2716108002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0fcce0 msgr2=0x7f9a5c10fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a59d9b700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0fcce0 0x7f9a5c10fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a59d9b700 1 -- 172.21.15.67:0/2716108002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50005040 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a59d9b700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c110140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009750 tx=0x7f9a50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50007c80 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a5000f040 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c005040 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.447+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50013c60 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.448+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c1180e0 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.448+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c108520 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.450+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9a5000e840 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.452+0000 7f9a4affd700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a4409b770 0x7f9a4409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:55.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.452+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f9a500c4f70 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.452+0000 7f9a5a59c700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a4409b770 0x7f9a4409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:55.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.453+0000 7f9a5a59c700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a4409b770 0x7f9a4409dc20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c004890 tx=0x7f9a4c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:55.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.456+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9a50090540 con 0x7f9a5c0fdef0 2023-10-07T23:15:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.658+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a5c1048e0 con 0x7f9a4409b770 2023-10-07T23:15:55.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.671+0000 7f9a4affd700 1 -- 172.21.15.67:0/2716108002 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423010 (secure 0 0 0) 0x7f9a5c1048e0 con 0x7f9a4409b770 2023-10-07T23:15:55.677 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:15:55.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a4409b770 msgr2=0x7f9a4409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a4409b770 0x7f9a4409dc20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c004890 tx=0x7f9a4c009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 msgr2=0x7f9a5c110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c110140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009750 tx=0x7f9a50009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 shutdown_connections 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0fcce0 0x7f9a5c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a4409b770 0x7f9a4409dc20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a5c0fe880 0x7f9a5c117190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 --2- 172.21.15.67:0/2716108002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a5c0fdef0 0x7f9a5c110140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:55.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 >> 172.21.15.67:0/2716108002 conn(0x7f9a5c0f8460 msgr2=0x7f9a5c0fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:55.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 shutdown_connections 2023-10-07T23:15:55.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:55.676+0000 7f9a5b59e700 1 -- 172.21.15.67:0/2716108002 wait complete. 2023-10-07T23:15:55.682 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:15:55.694 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-10-07T23:15:53.926544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596385,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":286,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3580792,"kb_used_data":13336,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121361032,"statfs":{"total":1151940427776,"available":1148273696768,"internally_reserved":0,"allocated":13656064,"data_stored":7491265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3711,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003420"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365769+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:15:31.365769+0000","last_peered":"2023-10-07T23:15:31.365769+0000","last_clean":"2023-10-07T23:15:31.365769+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:15:31.365769+0000","last_undegraded":"2023-10-07T23:15:31.365769+0000","last_fullsized":"2023-10-07T23:15:31.365769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.379445+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:15:31.379445+0000","last_peered":"2023-10-07T23:15:31.379445+0000","last_clean":"2023-10-07T23:15:31.379445+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:15:31.379445+0000","last_undegraded":"2023-10-07T23:15:31.379445+0000","last_fullsized":"2023-10-07T23:15:31.379445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.377008+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:15:31.377008+0000","last_peered":"2023-10-07T23:15:31.377008+0000","last_clean":"2023-10-07T23:15:31.377008+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:15:31.377008+0000","last_undegraded":"2023-10-07T23:15:31.377008+0000","last_fullsized":"2023-10-07T23:15:31.377008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.366980+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:15:31.366980+0000","last_peered":"2023-10-07T23:15:31.366980+0000","last_clean":"2023-10-07T23:15:31.366980+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:15:31.366980+0000","last_undegraded":"2023-10-07T23:15:31.366980+0000","last_fullsized":"2023-10-07T23:15:31.366980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370125+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:15:31.370125+0000","last_peered":"2023-10-07T23:15:31.370125+0000","last_clean":"2023-10-07T23:15:31.370125+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:15:31.370125+0000","last_undegraded":"2023-10-07T23:15:31.370125+0000","last_fullsized":"2023-10-07T23:15:31.370125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620266+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:15:31.620266+0000","last_peered":"2023-10-07T23:15:31.620266+0000","last_clean":"2023-10-07T23:15:31.620266+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:15:31.620266+0000","last_undegraded":"2023-10-07T23:15:31.620266+0000","last_fullsized":"2023-10-07T23:15:31.620266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371682+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:15:31.371682+0000","last_peered":"2023-10-07T23:15:31.371682+0000","last_clean":"2023-10-07T23:15:31.371682+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:15:31.371682+0000","last_undegraded":"2023-10-07T23:15:31.371682+0000","last_fullsized":"2023-10-07T23:15:31.371682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372800+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:15:31.372800+0000","last_peered":"2023-10-07T23:15:31.372800+0000","last_clean":"2023-10-07T23:15:31.372800+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:15:31.372800+0000","last_undegraded":"2023-10-07T23:15:31.372800+0000","last_fullsized":"2023-10-07T23:15:31.372800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372605+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:15:31.372605+0000","last_peered":"2023-10-07T23:15:31.372605+0000","last_clean":"2023-10-07T23:15:31.372605+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:15:31.372605+0000","last_undegraded":"2023-10-07T23:15:31.372605+0000","last_fullsized":"2023-10-07T23:15:31.372605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620876+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:15:31.620876+0000","last_peered":"2023-10-07T23:15:31.620876+0000","last_clean":"2023-10-07T23:15:31.620876+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:15:31.620876+0000","last_undegraded":"2023-10-07T23:15:31.620876+0000","last_fullsized":"2023-10-07T23:15:31.620876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372347+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:15:31.372347+0000","last_peered":"2023-10-07T23:15:31.372347+0000","last_clean":"2023-10-07T23:15:31.372347+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:15:31.372347+0000","last_undegraded":"2023-10-07T23:15:31.372347+0000","last_fullsized":"2023-10-07T23:15:31.372347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.366879+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:15:31.366879+0000","last_peered":"2023-10-07T23:15:31.366879+0000","last_clean":"2023-10-07T23:15:31.366879+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:15:31.366879+0000","last_undegraded":"2023-10-07T23:15:31.366879+0000","last_fullsized":"2023-10-07T23:15:31.366879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369784+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:15:31.369784+0000","last_peered":"2023-10-07T23:15:31.369784+0000","last_clean":"2023-10-07T23:15:31.369784+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:15:31.369784+0000","last_undegraded":"2023-10-07T23:15:31.369784+0000","last_fullsized":"2023-10-07T23:15:31.369784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376755+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:15:31.376755+0000","last_peered":"2023-10-07T23:15:31.376755+0000","last_clean":"2023-10-07T23:15:31.376755+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:15:31.376755+0000","last_undegraded":"2023-10-07T23:15:31.376755+0000","last_fullsized":"2023-10-07T23:15:31.376755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.383512+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:15:31.383512+0000","last_peered":"2023-10-07T23:15:31.383512+0000","last_clean":"2023-10-07T23:15:31.383512+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:15:31.383512+0000","last_undegraded":"2023-10-07T23:15:31.383512+0000","last_fullsized":"2023-10-07T23:15:31.383512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371531+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:15:31.371531+0000","last_peered":"2023-10-07T23:15:31.371531+0000","last_clean":"2023-10-07T23:15:31.371531+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:15:31.371531+0000","last_undegraded":"2023-10-07T23:15:31.371531+0000","last_fullsized":"2023-10-07T23:15:31.371531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365916+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:15:31.365916+0000","last_peered":"2023-10-07T23:15:31.365916+0000","last_clean":"2023-10-07T23:15:31.365916+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:15:31.365916+0000","last_undegraded":"2023-10-07T23:15:31.365916+0000","last_fullsized":"2023-10-07T23:15:31.365916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.377279+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:15:31.377279+0000","last_peered":"2023-10-07T23:15:31.377279+0000","last_clean":"2023-10-07T23:15:31.377279+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:15:31.377279+0000","last_undegraded":"2023-10-07T23:15:31.377279+0000","last_fullsized":"2023-10-07T23:15:31.377279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374813+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:15:31.374813+0000","last_peered":"2023-10-07T23:15:31.374813+0000","last_clean":"2023-10-07T23:15:31.374813+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:15:31.374813+0000","last_undegraded":"2023-10-07T23:15:31.374813+0000","last_fullsized":"2023-10-07T23:15:31.374813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.389673+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:15:31.389673+0000","last_peered":"2023-10-07T23:15:31.389673+0000","last_clean":"2023-10-07T23:15:31.389673+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:15:31.389673+0000","last_undegraded":"2023-10-07T23:15:31.389673+0000","last_fullsized":"2023-10-07T23:15:31.389673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.023841+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:15:53.023841+0000","last_peered":"2023-10-07T23:15:53.023841+0000","last_clean":"2023-10-07T23:15:53.023841+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:15:53.023841+0000","last_undegraded":"2023-10-07T23:15:53.023841+0000","last_fullsized":"2023-10-07T23:15:53.023841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373919+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:15:31.373919+0000","last_peered":"2023-10-07T23:15:31.373919+0000","last_clean":"2023-10-07T23:15:31.373919+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:15:31.373919+0000","last_undegraded":"2023-10-07T23:15:31.373919+0000","last_fullsized":"2023-10-07T23:15:31.373919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386744+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:15:31.386744+0000","last_peered":"2023-10-07T23:15:31.386744+0000","last_clean":"2023-10-07T23:15:31.386744+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:15:31.386744+0000","last_undegraded":"2023-10-07T23:15:31.386744+0000","last_fullsized":"2023-10-07T23:15:31.386744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374006+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:15:31.374006+0000","last_peered":"2023-10-07T23:15:31.374006+0000","last_clean":"2023-10-07T23:15:31.374006+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:15:31.374006+0000","last_undegraded":"2023-10-07T23:15:31.374006+0000","last_fullsized":"2023-10-07T23:15:31.374006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380432+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:15:31.380432+0000","last_peered":"2023-10-07T23:15:31.380432+0000","last_clean":"2023-10-07T23:15:31.380432+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:15:31.380432+0000","last_undegraded":"2023-10-07T23:15:31.380432+0000","last_fullsized":"2023-10-07T23:15:31.380432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.389586+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:15:31.389586+0000","last_peered":"2023-10-07T23:15:31.389586+0000","last_clean":"2023-10-07T23:15:31.389586+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:15:31.389586+0000","last_undegraded":"2023-10-07T23:15:31.389586+0000","last_fullsized":"2023-10-07T23:15:31.389586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.020240+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:15:53.020240+0000","last_peered":"2023-10-07T23:15:53.020240+0000","last_clean":"2023-10-07T23:15:53.020240+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:15:53.020240+0000","last_undegraded":"2023-10-07T23:15:53.020240+0000","last_fullsized":"2023-10-07T23:15:53.020240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372782+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:15:31.372782+0000","last_peered":"2023-10-07T23:15:31.372782+0000","last_clean":"2023-10-07T23:15:31.372782+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:15:31.372782+0000","last_undegraded":"2023-10-07T23:15:31.372782+0000","last_fullsized":"2023-10-07T23:15:31.372782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386725+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:15:31.386725+0000","last_peered":"2023-10-07T23:15:31.386725+0000","last_clean":"2023-10-07T23:15:31.386725+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:15:31.386725+0000","last_undegraded":"2023-10-07T23:15:31.386725+0000","last_fullsized":"2023-10-07T23:15:31.386725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370386+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:15:31.370386+0000","last_peered":"2023-10-07T23:15:31.370386+0000","last_clean":"2023-10-07T23:15:31.370386+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:15:31.370386+0000","last_undegraded":"2023-10-07T23:15:31.370386+0000","last_fullsized":"2023-10-07T23:15:31.370386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376424+0000","last_change":"2023-10-07T23:15:27.197818+0000","last_active":"2023-10-07T23:15:31.376424+0000","last_peered":"2023-10-07T23:15:31.376424+0000","last_clean":"2023-10-07T23:15:31.376424+0000","last_became_active":"2023-10-07T23:15:27.197522+0000","last_became_peered":"2023-10-07T23:15:27.197522+0000","last_unstale":"2023-10-07T23:15:31.376424+0000","last_undegraded":"2023-10-07T23:15:31.376424+0000","last_fullsized":"2023-10-07T23:15:31.376424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:11.128855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380564+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:15:31.380564+0000","last_peered":"2023-10-07T23:15:31.380564+0000","last_clean":"2023-10-07T23:15:31.380564+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:15:31.380564+0000","last_undegraded":"2023-10-07T23:15:31.380564+0000","last_fullsized":"2023-10-07T23:15:31.380564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.017175+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:15:52.017175+0000","last_peered":"2023-10-07T23:15:52.017175+0000","last_clean":"2023-10-07T23:15:52.017175+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:15:52.017175+0000","last_undegraded":"2023-10-07T23:15:52.017175+0000","last_fullsized":"2023-10-07T23:15:52.017175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370472+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:15:31.370472+0000","last_peered":"2023-10-07T23:15:31.370472+0000","last_clean":"2023-10-07T23:15:31.370472+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:15:31.370472+0000","last_undegraded":"2023-10-07T23:15:31.370472+0000","last_fullsized":"2023-10-07T23:15:31.370472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.387209+0000","last_change":"2023-10-07T23:15:31.386751+0000","last_active":"2023-10-07T23:15:31.387209+0000","last_peered":"2023-10-07T23:15:31.387209+0000","last_clean":"2023-10-07T23:15:31.387209+0000","last_became_active":"2023-10-07T23:15:31.386370+0000","last_became_peered":"2023-10-07T23:15:31.386370+0000","last_unstale":"2023-10-07T23:15:31.387209+0000","last_undegraded":"2023-10-07T23:15:31.387209+0000","last_fullsized":"2023-10-07T23:15:31.387209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:26:43.668892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620283+0000","last_change":"2023-10-07T23:15:26.349502+0000","last_active":"2023-10-07T23:15:31.620283+0000","last_peered":"2023-10-07T23:15:31.620283+0000","last_clean":"2023-10-07T23:15:31.620283+0000","last_became_active":"2023-10-07T23:15:26.349182+0000","last_became_peered":"2023-10-07T23:15:26.349182+0000","last_unstale":"2023-10-07T23:15:31.620283+0000","last_undegraded":"2023-10-07T23:15:31.620283+0000","last_fullsized":"2023-10-07T23:15:31.620283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:40.762251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.375571+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:15:31.375571+0000","last_peered":"2023-10-07T23:15:31.375571+0000","last_clean":"2023-10-07T23:15:31.375571+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:15:31.375571+0000","last_undegraded":"2023-10-07T23:15:31.375571+0000","last_fullsized":"2023-10-07T23:15:31.375571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.393275+0000","last_change":"2023-10-07T23:15:31.392966+0000","last_active":"2023-10-07T23:15:31.393275+0000","last_peered":"2023-10-07T23:15:31.393275+0000","last_clean":"2023-10-07T23:15:31.393275+0000","last_became_active":"2023-10-07T23:15:31.392612+0000","last_became_peered":"2023-10-07T23:15:31.392612+0000","last_unstale":"2023-10-07T23:15:31.393275+0000","last_undegraded":"2023-10-07T23:15:31.393275+0000","last_fullsized":"2023-10-07T23:15:31.393275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:09:36.871330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.023716+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:15:53.023716+0000","last_peered":"2023-10-07T23:15:53.023716+0000","last_clean":"2023-10-07T23:15:53.023716+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:15:53.023716+0000","last_undegraded":"2023-10-07T23:15:53.023716+0000","last_fullsized":"2023-10-07T23:15:53.023716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372903+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:15:31.372903+0000","last_peered":"2023-10-07T23:15:31.372903+0000","last_clean":"2023-10-07T23:15:31.372903+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:15:31.372903+0000","last_undegraded":"2023-10-07T23:15:31.372903+0000","last_fullsized":"2023-10-07T23:15:31.372903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.395981+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:15:31.395981+0000","last_peered":"2023-10-07T23:15:31.395981+0000","last_clean":"2023-10-07T23:15:31.395981+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:15:31.395981+0000","last_undegraded":"2023-10-07T23:15:31.395981+0000","last_fullsized":"2023-10-07T23:15:31.395981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.348021+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:15:34.348021+0000","last_peered":"2023-10-07T23:15:34.348021+0000","last_clean":"2023-10-07T23:15:34.348021+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:15:34.348021+0000","last_undegraded":"2023-10-07T23:15:34.348021+0000","last_fullsized":"2023-10-07T23:15:34.348021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376554+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:15:31.376554+0000","last_peered":"2023-10-07T23:15:31.376554+0000","last_clean":"2023-10-07T23:15:31.376554+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:15:31.376554+0000","last_undegraded":"2023-10-07T23:15:31.376554+0000","last_fullsized":"2023-10-07T23:15:31.376554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.385921+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:15:31.385921+0000","last_peered":"2023-10-07T23:15:31.385921+0000","last_clean":"2023-10-07T23:15:31.385921+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:15:31.385921+0000","last_undegraded":"2023-10-07T23:15:31.385921+0000","last_fullsized":"2023-10-07T23:15:31.385921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.015458+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:15:52.015458+0000","last_peered":"2023-10-07T23:15:52.015458+0000","last_clean":"2023-10-07T23:15:52.015458+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:15:52.015458+0000","last_undegraded":"2023-10-07T23:15:52.015458+0000","last_fullsized":"2023-10-07T23:15:52.015458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372294+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:15:31.372294+0000","last_peered":"2023-10-07T23:15:31.372294+0000","last_clean":"2023-10-07T23:15:31.372294+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:15:31.372294+0000","last_undegraded":"2023-10-07T23:15:31.372294+0000","last_fullsized":"2023-10-07T23:15:31.372294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.394523+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:15:31.394523+0000","last_peered":"2023-10-07T23:15:31.394523+0000","last_clean":"2023-10-07T23:15:31.394523+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:15:31.394523+0000","last_undegraded":"2023-10-07T23:15:31.394523+0000","last_fullsized":"2023-10-07T23:15:31.394523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016581+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:15:52.016581+0000","last_peered":"2023-10-07T23:15:52.016581+0000","last_clean":"2023-10-07T23:15:52.016581+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:15:52.016581+0000","last_undegraded":"2023-10-07T23:15:52.016581+0000","last_fullsized":"2023-10-07T23:15:52.016581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380519+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:15:31.380519+0000","last_peered":"2023-10-07T23:15:31.380519+0000","last_clean":"2023-10-07T23:15:31.380519+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:15:31.380519+0000","last_undegraded":"2023-10-07T23:15:31.380519+0000","last_fullsized":"2023-10-07T23:15:31.380519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386111+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:15:31.386111+0000","last_peered":"2023-10-07T23:15:31.386111+0000","last_clean":"2023-10-07T23:15:31.386111+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:15:31.386111+0000","last_undegraded":"2023-10-07T23:15:31.386111+0000","last_fullsized":"2023-10-07T23:15:31.386111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.347613+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:15:34.347613+0000","last_peered":"2023-10-07T23:15:34.347613+0000","last_clean":"2023-10-07T23:15:34.347613+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:15:34.347613+0000","last_undegraded":"2023-10-07T23:15:34.347613+0000","last_fullsized":"2023-10-07T23:15:34.347613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373002+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:15:31.373002+0000","last_peered":"2023-10-07T23:15:31.373002+0000","last_clean":"2023-10-07T23:15:31.373002+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:15:31.373002+0000","last_undegraded":"2023-10-07T23:15:31.373002+0000","last_fullsized":"2023-10-07T23:15:31.373002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.391512+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:15:31.391512+0000","last_peered":"2023-10-07T23:15:31.391512+0000","last_clean":"2023-10-07T23:15:31.391512+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:15:31.391512+0000","last_undegraded":"2023-10-07T23:15:31.391512+0000","last_fullsized":"2023-10-07T23:15:31.391512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016432+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:15:52.016432+0000","last_peered":"2023-10-07T23:15:52.016432+0000","last_clean":"2023-10-07T23:15:52.016432+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:15:52.016432+0000","last_undegraded":"2023-10-07T23:15:52.016432+0000","last_fullsized":"2023-10-07T23:15:52.016432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370864+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:15:31.370864+0000","last_peered":"2023-10-07T23:15:31.370864+0000","last_clean":"2023-10-07T23:15:31.370864+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:15:31.370864+0000","last_undegraded":"2023-10-07T23:15:31.370864+0000","last_fullsized":"2023-10-07T23:15:31.370864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386772+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:15:31.386772+0000","last_peered":"2023-10-07T23:15:31.386772+0000","last_clean":"2023-10-07T23:15:31.386772+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:15:31.386772+0000","last_undegraded":"2023-10-07T23:15:31.386772+0000","last_fullsized":"2023-10-07T23:15:31.386772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374111+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:15:31.374111+0000","last_peered":"2023-10-07T23:15:31.374111+0000","last_clean":"2023-10-07T23:15:31.374111+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:15:31.374111+0000","last_undegraded":"2023-10-07T23:15:31.374111+0000","last_fullsized":"2023-10-07T23:15:31.374111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.368794+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:15:31.368794+0000","last_peered":"2023-10-07T23:15:31.368794+0000","last_clean":"2023-10-07T23:15:31.368794+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:15:31.368794+0000","last_undegraded":"2023-10-07T23:15:31.368794+0000","last_fullsized":"2023-10-07T23:15:31.368794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.385305+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:15:31.385305+0000","last_peered":"2023-10-07T23:15:31.385305+0000","last_clean":"2023-10-07T23:15:31.385305+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:15:31.385305+0000","last_undegraded":"2023-10-07T23:15:31.385305+0000","last_fullsized":"2023-10-07T23:15:31.385305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.017231+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:15:52.017231+0000","last_peered":"2023-10-07T23:15:52.017231+0000","last_clean":"2023-10-07T23:15:52.017231+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:15:52.017231+0000","last_undegraded":"2023-10-07T23:15:52.017231+0000","last_fullsized":"2023-10-07T23:15:52.017231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372062+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:15:31.372062+0000","last_peered":"2023-10-07T23:15:31.372062+0000","last_clean":"2023-10-07T23:15:31.372062+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:15:31.372062+0000","last_undegraded":"2023-10-07T23:15:31.372062+0000","last_fullsized":"2023-10-07T23:15:31.372062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.398606+0000","last_change":"2023-10-07T23:15:31.397451+0000","last_active":"2023-10-07T23:15:31.398606+0000","last_peered":"2023-10-07T23:15:31.398606+0000","last_clean":"2023-10-07T23:15:31.398606+0000","last_became_active":"2023-10-07T23:15:31.397001+0000","last_became_peered":"2023-10-07T23:15:31.397001+0000","last_unstale":"2023-10-07T23:15:31.398606+0000","last_undegraded":"2023-10-07T23:15:31.398606+0000","last_fullsized":"2023-10-07T23:15:31.398606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:35:45.208477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365152+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:15:31.365152+0000","last_peered":"2023-10-07T23:15:31.365152+0000","last_clean":"2023-10-07T23:15:31.365152+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:15:31.365152+0000","last_undegraded":"2023-10-07T23:15:31.365152+0000","last_fullsized":"2023-10-07T23:15:31.365152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371467+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:15:31.371467+0000","last_peered":"2023-10-07T23:15:31.371467+0000","last_clean":"2023-10-07T23:15:31.371467+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:15:31.371467+0000","last_undegraded":"2023-10-07T23:15:31.371467+0000","last_fullsized":"2023-10-07T23:15:31.371467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.393511+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:15:31.393511+0000","last_peered":"2023-10-07T23:15:31.393511+0000","last_clean":"2023-10-07T23:15:31.393511+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:15:31.393511+0000","last_undegraded":"2023-10-07T23:15:31.393511+0000","last_fullsized":"2023-10-07T23:15:31.393511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.022068+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:15:53.022068+0000","last_peered":"2023-10-07T23:15:53.022068+0000","last_clean":"2023-10-07T23:15:53.022068+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:15:53.022068+0000","last_undegraded":"2023-10-07T23:15:53.022068+0000","last_fullsized":"2023-10-07T23:15:53.022068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376151+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:15:31.376151+0000","last_peered":"2023-10-07T23:15:31.376151+0000","last_clean":"2023-10-07T23:15:31.376151+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:15:31.376151+0000","last_undegraded":"2023-10-07T23:15:31.376151+0000","last_fullsized":"2023-10-07T23:15:31.376151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.389661+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:15:31.389661+0000","last_peered":"2023-10-07T23:15:31.389661+0000","last_clean":"2023-10-07T23:15:31.389661+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:15:31.389661+0000","last_undegraded":"2023-10-07T23:15:31.389661+0000","last_fullsized":"2023-10-07T23:15:31.389661+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370594+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:15:31.370594+0000","last_peered":"2023-10-07T23:15:31.370594+0000","last_clean":"2023-10-07T23:15:31.370594+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:15:31.370594+0000","last_undegraded":"2023-10-07T23:15:31.370594+0000","last_fullsized":"2023-10-07T23:15:31.370594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365124+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:15:31.365124+0000","last_peered":"2023-10-07T23:15:31.365124+0000","last_clean":"2023-10-07T23:15:31.365124+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:15:31.365124+0000","last_undegraded":"2023-10-07T23:15:31.365124+0000","last_fullsized":"2023-10-07T23:15:31.365124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.622769+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:15:31.622769+0000","last_peered":"2023-10-07T23:15:31.622769+0000","last_clean":"2023-10-07T23:15:31.622769+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:15:31.622769+0000","last_undegraded":"2023-10-07T23:15:31.622769+0000","last_fullsized":"2023-10-07T23:15:31.622769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376822+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:15:31.376822+0000","last_peered":"2023-10-07T23:15:31.376822+0000","last_clean":"2023-10-07T23:15:31.376822+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:15:31.376822+0000","last_undegraded":"2023-10-07T23:15:31.376822+0000","last_fullsized":"2023-10-07T23:15:31.376822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620719+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:15:31.620719+0000","last_peered":"2023-10-07T23:15:31.620719+0000","last_clean":"2023-10-07T23:15:31.620719+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:15:31.620719+0000","last_undegraded":"2023-10-07T23:15:31.620719+0000","last_fullsized":"2023-10-07T23:15:31.620719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.396051+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:15:31.396051+0000","last_peered":"2023-10-07T23:15:31.396051+0000","last_clean":"2023-10-07T23:15:31.396051+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:15:31.396051+0000","last_undegraded":"2023-10-07T23:15:31.396051+0000","last_fullsized":"2023-10-07T23:15:31.396051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372517+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:15:31.372517+0000","last_peered":"2023-10-07T23:15:31.372517+0000","last_clean":"2023-10-07T23:15:31.372517+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:15:31.372517+0000","last_undegraded":"2023-10-07T23:15:31.372517+0000","last_fullsized":"2023-10-07T23:15:31.372517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.620703+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:15:31.620703+0000","last_peered":"2023-10-07T23:15:31.620703+0000","last_clean":"2023-10-07T23:15:31.620703+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:15:31.620703+0000","last_undegraded":"2023-10-07T23:15:31.620703+0000","last_fullsized":"2023-10-07T23:15:31.620703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623366+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:15:31.623366+0000","last_peered":"2023-10-07T23:15:31.623366+0000","last_clean":"2023-10-07T23:15:31.623366+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:15:31.623366+0000","last_undegraded":"2023-10-07T23:15:31.623366+0000","last_fullsized":"2023-10-07T23:15:31.623366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376698+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:15:31.376698+0000","last_peered":"2023-10-07T23:15:31.376698+0000","last_clean":"2023-10-07T23:15:31.376698+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:15:31.376698+0000","last_undegraded":"2023-10-07T23:15:31.376698+0000","last_fullsized":"2023-10-07T23:15:31.376698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.022743+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:15:53.022743+0000","last_peered":"2023-10-07T23:15:53.022743+0000","last_clean":"2023-10-07T23:15:53.022743+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:15:53.022743+0000","last_undegraded":"2023-10-07T23:15:53.022743+0000","last_fullsized":"2023-10-07T23:15:53.022743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.386149+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:15:31.386149+0000","last_peered":"2023-10-07T23:15:31.386149+0000","last_clean":"2023-10-07T23:15:31.386149+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:15:31.386149+0000","last_undegraded":"2023-10-07T23:15:31.386149+0000","last_fullsized":"2023-10-07T23:15:31.386149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.375648+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:15:31.375648+0000","last_peered":"2023-10-07T23:15:31.375648+0000","last_clean":"2023-10-07T23:15:31.375648+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:15:31.375648+0000","last_undegraded":"2023-10-07T23:15:31.375648+0000","last_fullsized":"2023-10-07T23:15:31.375648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016958+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:15:52.016958+0000","last_peered":"2023-10-07T23:15:52.016958+0000","last_clean":"2023-10-07T23:15:52.016958+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:15:52.016958+0000","last_undegraded":"2023-10-07T23:15:52.016958+0000","last_fullsized":"2023-10-07T23:15:52.016958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370819+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:15:31.370819+0000","last_peered":"2023-10-07T23:15:31.370819+0000","last_clean":"2023-10-07T23:15:31.370819+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:15:31.370819+0000","last_undegraded":"2023-10-07T23:15:31.370819+0000","last_fullsized":"2023-10-07T23:15:31.370819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623248+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:15:31.623248+0000","last_peered":"2023-10-07T23:15:31.623248+0000","last_clean":"2023-10-07T23:15:31.623248+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:15:31.623248+0000","last_undegraded":"2023-10-07T23:15:31.623248+0000","last_fullsized":"2023-10-07T23:15:31.623248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.367855+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:15:31.367855+0000","last_peered":"2023-10-07T23:15:31.367855+0000","last_clean":"2023-10-07T23:15:31.367855+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:15:31.367855+0000","last_undegraded":"2023-10-07T23:15:31.367855+0000","last_fullsized":"2023-10-07T23:15:31.367855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.214102+0000","last_change":"2023-10-07T23:15:26.352454+0000","last_active":"2023-10-07T23:15:34.214102+0000","last_peered":"2023-10-07T23:15:34.214102+0000","last_clean":"2023-10-07T23:15:34.214102+0000","last_became_active":"2023-10-07T23:15:26.352156+0000","last_became_peered":"2023-10-07T23:15:26.352156+0000","last_unstale":"2023-10-07T23:15:34.214102+0000","last_undegraded":"2023-10-07T23:15:34.214102+0000","last_fullsized":"2023-10-07T23:15:34.214102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:21.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380182+0000","last_change":"2023-10-07T23:15:31.379007+0000","last_active":"2023-10-07T23:15:31.380182+0000","last_peered":"2023-10-07T23:15:31.380182+0000","last_clean":"2023-10-07T23:15:31.380182+0000","last_became_active":"2023-10-07T23:15:31.378549+0000","last_became_peered":"2023-10-07T23:15:31.378549+0000","last_unstale":"2023-10-07T23:15:31.380182+0000","last_undegraded":"2023-10-07T23:15:31.380182+0000","last_fullsized":"2023-10-07T23:15:31.380182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:47.745473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:34.348255+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:15:34.348255+0000","last_peered":"2023-10-07T23:15:34.348255+0000","last_clean":"2023-10-07T23:15:34.348255+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:15:34.348255+0000","last_undegraded":"2023-10-07T23:15:34.348255+0000","last_fullsized":"2023-10-07T23:15:34.348255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.368777+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:15:31.368777+0000","last_peered":"2023-10-07T23:15:31.368777+0000","last_clean":"2023-10-07T23:15:31.368777+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:15:31.368777+0000","last_undegraded":"2023-10-07T23:15:31.368777+0000","last_fullsized":"2023-10-07T23:15:31.368777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623563+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:15:31.623563+0000","last_peered":"2023-10-07T23:15:31.623563+0000","last_clean":"2023-10-07T23:15:31.623563+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:15:31.623563+0000","last_undegraded":"2023-10-07T23:15:31.623563+0000","last_fullsized":"2023-10-07T23:15:31.623563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:53.023306+0000","last_change":"2023-10-07T23:15:26.363682+0000","last_active":"2023-10-07T23:15:53.023306+0000","last_peered":"2023-10-07T23:15:53.023306+0000","last_clean":"2023-10-07T23:15:53.023306+0000","last_became_active":"2023-10-07T23:15:26.362955+0000","last_became_peered":"2023-10-07T23:15:26.362955+0000","last_unstale":"2023-10-07T23:15:53.023306+0000","last_undegraded":"2023-10-07T23:15:53.023306+0000","last_fullsized":"2023-10-07T23:15:53.023306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:48:41.150623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371255+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:15:31.371255+0000","last_peered":"2023-10-07T23:15:31.371255+0000","last_clean":"2023-10-07T23:15:31.371255+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:15:31.371255+0000","last_undegraded":"2023-10-07T23:15:31.371255+0000","last_fullsized":"2023-10-07T23:15:31.371255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380428+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:15:31.380428+0000","last_peered":"2023-10-07T23:15:31.380428+0000","last_clean":"2023-10-07T23:15:31.380428+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:15:31.380428+0000","last_undegraded":"2023-10-07T23:15:31.380428+0000","last_fullsized":"2023-10-07T23:15:31.380428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016888+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:15:52.016888+0000","last_peered":"2023-10-07T23:15:52.016888+0000","last_clean":"2023-10-07T23:15:52.016888+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:15:52.016888+0000","last_undegraded":"2023-10-07T23:15:52.016888+0000","last_fullsized":"2023-10-07T23:15:52.016888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.379537+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:15:31.379537+0000","last_peered":"2023-10-07T23:15:31.379537+0000","last_clean":"2023-10-07T23:15:31.379537+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:15:31.379537+0000","last_undegraded":"2023-10-07T23:15:31.379537+0000","last_fullsized":"2023-10-07T23:15:31.379537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623340+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:15:31.623340+0000","last_peered":"2023-10-07T23:15:31.623340+0000","last_clean":"2023-10-07T23:15:31.623340+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:15:31.623340+0000","last_undegraded":"2023-10-07T23:15:31.623340+0000","last_fullsized":"2023-10-07T23:15:31.623340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373136+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:15:31.373136+0000","last_peered":"2023-10-07T23:15:31.373136+0000","last_clean":"2023-10-07T23:15:31.373136+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:15:31.373136+0000","last_undegraded":"2023-10-07T23:15:31.373136+0000","last_fullsized":"2023-10-07T23:15:31.373136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.370060+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:15:31.370060+0000","last_peered":"2023-10-07T23:15:31.370060+0000","last_clean":"2023-10-07T23:15:31.370060+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:15:31.370060+0000","last_undegraded":"2023-10-07T23:15:31.370060+0000","last_fullsized":"2023-10-07T23:15:31.370060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.397249+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:15:31.397249+0000","last_peered":"2023-10-07T23:15:31.397249+0000","last_clean":"2023-10-07T23:15:31.397249+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:15:31.397249+0000","last_undegraded":"2023-10-07T23:15:31.397249+0000","last_fullsized":"2023-10-07T23:15:31.397249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.017471+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:15:52.017471+0000","last_peered":"2023-10-07T23:15:52.017471+0000","last_clean":"2023-10-07T23:15:52.017471+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:15:52.017471+0000","last_undegraded":"2023-10-07T23:15:52.017471+0000","last_fullsized":"2023-10-07T23:15:52.017471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.364899+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:15:31.364899+0000","last_peered":"2023-10-07T23:15:31.364899+0000","last_clean":"2023-10-07T23:15:31.364899+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:15:31.364899+0000","last_undegraded":"2023-10-07T23:15:31.364899+0000","last_fullsized":"2023-10-07T23:15:31.364899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.385895+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:15:31.385895+0000","last_peered":"2023-10-07T23:15:31.385895+0000","last_clean":"2023-10-07T23:15:31.385895+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:15:31.385895+0000","last_undegraded":"2023-10-07T23:15:31.385895+0000","last_fullsized":"2023-10-07T23:15:31.385895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373249+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:15:31.373249+0000","last_peered":"2023-10-07T23:15:31.373249+0000","last_clean":"2023-10-07T23:15:31.373249+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:15:31.373249+0000","last_undegraded":"2023-10-07T23:15:31.373249+0000","last_fullsized":"2023-10-07T23:15:31.373249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.378191+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:15:31.378191+0000","last_peered":"2023-10-07T23:15:31.378191+0000","last_clean":"2023-10-07T23:15:31.378191+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:15:31.378191+0000","last_undegraded":"2023-10-07T23:15:31.378191+0000","last_fullsized":"2023-10-07T23:15:31.378191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.623298+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:15:31.623298+0000","last_peered":"2023-10-07T23:15:31.623298+0000","last_clean":"2023-10-07T23:15:31.623298+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:15:31.623298+0000","last_undegraded":"2023-10-07T23:15:31.623298+0000","last_fullsized":"2023-10-07T23:15:31.623298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-07T23:15:52.016419+0000","last_change":"2023-10-07T23:15:26.357820+0000","last_active":"2023-10-07T23:15:52.016419+0000","last_peered":"2023-10-07T23:15:52.016419+0000","last_clean":"2023-10-07T23:15:52.016419+0000","last_became_active":"2023-10-07T23:15:26.357545+0000","last_became_peered":"2023-10-07T23:15:26.357545+0000","last_unstale":"2023-10-07T23:15:52.016419+0000","last_undegraded":"2023-10-07T23:15:52.016419+0000","last_fullsized":"2023-10-07T23:15:52.016419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:25:15.446891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372269+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:15:31.372269+0000","last_peered":"2023-10-07T23:15:31.372269+0000","last_clean":"2023-10-07T23:15:31.372269+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:15:31.372269+0000","last_undegraded":"2023-10-07T23:15:31.372269+0000","last_fullsized":"2023-10-07T23:15:31.372269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380899+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:15:31.380899+0000","last_peered":"2023-10-07T23:15:31.380899+0000","last_clean":"2023-10-07T23:15:31.380899+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:15:31.380899+0000","last_undegraded":"2023-10-07T23:15:31.380899+0000","last_fullsized":"2023-10-07T23:15:31.380899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376215+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:15:31.376215+0000","last_peered":"2023-10-07T23:15:31.376215+0000","last_clean":"2023-10-07T23:15:31.376215+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:15:31.376215+0000","last_undegraded":"2023-10-07T23:15:31.376215+0000","last_fullsized":"2023-10-07T23:15:31.376215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372429+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:15:31.372429+0000","last_peered":"2023-10-07T23:15:31.372429+0000","last_clean":"2023-10-07T23:15:31.372429+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:15:31.372429+0000","last_undegraded":"2023-10-07T23:15:31.372429+0000","last_fullsized":"2023-10-07T23:15:31.372429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.396751+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:15:31.396751+0000","last_peered":"2023-10-07T23:15:31.396751+0000","last_clean":"2023-10-07T23:15:31.396751+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:15:31.396751+0000","last_undegraded":"2023-10-07T23:15:31.396751+0000","last_fullsized":"2023-10-07T23:15:31.396751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376104+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:15:31.376104+0000","last_peered":"2023-10-07T23:15:31.376104+0000","last_clean":"2023-10-07T23:15:31.376104+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:15:31.376104+0000","last_undegraded":"2023-10-07T23:15:31.376104+0000","last_fullsized":"2023-10-07T23:15:31.376104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371831+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:15:31.371831+0000","last_peered":"2023-10-07T23:15:31.371831+0000","last_clean":"2023-10-07T23:15:31.371831+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:15:31.371831+0000","last_undegraded":"2023-10-07T23:15:31.371831+0000","last_fullsized":"2023-10-07T23:15:31.371831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.380633+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:15:31.380633+0000","last_peered":"2023-10-07T23:15:31.380633+0000","last_clean":"2023-10-07T23:15:31.380633+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:15:31.380633+0000","last_undegraded":"2023-10-07T23:15:31.380633+0000","last_fullsized":"2023-10-07T23:15:31.380633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369993+0000","last_change":"2023-10-07T23:15:26.359589+0000","last_active":"2023-10-07T23:15:31.369993+0000","last_peered":"2023-10-07T23:15:31.369993+0000","last_clean":"2023-10-07T23:15:31.369993+0000","last_became_active":"2023-10-07T23:15:26.359222+0000","last_became_peered":"2023-10-07T23:15:26.359222+0000","last_unstale":"2023-10-07T23:15:31.369993+0000","last_undegraded":"2023-10-07T23:15:31.369993+0000","last_fullsized":"2023-10-07T23:15:31.369993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:09.393841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372074+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:15:31.372074+0000","last_peered":"2023-10-07T23:15:31.372074+0000","last_clean":"2023-10-07T23:15:31.372074+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:15:31.372074+0000","last_undegraded":"2023-10-07T23:15:31.372074+0000","last_fullsized":"2023-10-07T23:15:31.372074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.364826+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:15:31.364826+0000","last_peered":"2023-10-07T23:15:31.364826+0000","last_clean":"2023-10-07T23:15:31.364826+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:15:31.364826+0000","last_undegraded":"2023-10-07T23:15:31.364826+0000","last_fullsized":"2023-10-07T23:15:31.364826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.374782+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:15:31.374782+0000","last_peered":"2023-10-07T23:15:31.374782+0000","last_clean":"2023-10-07T23:15:31.374782+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:15:31.374782+0000","last_undegraded":"2023-10-07T23:15:31.374782+0000","last_fullsized":"2023-10-07T23:15:31.374782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373134+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:15:31.373134+0000","last_peered":"2023-10-07T23:15:31.373134+0000","last_clean":"2023-10-07T23:15:31.373134+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:15:31.373134+0000","last_undegraded":"2023-10-07T23:15:31.373134+0000","last_fullsized":"2023-10-07T23:15:31.373134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376784+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:15:31.376784+0000","last_peered":"2023-10-07T23:15:31.376784+0000","last_clean":"2023-10-07T23:15:31.376784+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:15:31.376784+0000","last_undegraded":"2023-10-07T23:15:31.376784+0000","last_fullsized":"2023-10-07T23:15:31.376784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369038+0000","last_change":"2023-10-07T23:15:27.206430+0000","last_active":"2023-10-07T23:15:31.369038+0000","last_peered":"2023-10-07T23:15:31.369038+0000","last_clean":"2023-10-07T23:15:31.369038+0000","last_became_active":"2023-10-07T23:15:27.206044+0000","last_became_peered":"2023-10-07T23:15:27.206044+0000","last_unstale":"2023-10-07T23:15:31.369038+0000","last_undegraded":"2023-10-07T23:15:31.369038+0000","last_fullsized":"2023-10-07T23:15:31.369038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:57.365683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365107+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:15:31.365107+0000","last_peered":"2023-10-07T23:15:31.365107+0000","last_clean":"2023-10-07T23:15:31.365107+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:15:31.365107+0000","last_undegraded":"2023-10-07T23:15:31.365107+0000","last_fullsized":"2023-10-07T23:15:31.365107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.373419+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:15:31.373419+0000","last_peered":"2023-10-07T23:15:31.373419+0000","last_clean":"2023-10-07T23:15:31.373419+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:15:31.373419+0000","last_undegraded":"2023-10-07T23:15:31.373419+0000","last_fullsized":"2023-10-07T23:15:31.373419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.371337+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:15:31.371337+0000","last_peered":"2023-10-07T23:15:31.371337+0000","last_clean":"2023-10-07T23:15:31.371337+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:15:31.371337+0000","last_undegraded":"2023-10-07T23:15:31.371337+0000","last_fullsized":"2023-10-07T23:15:31.371337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.376665+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:15:31.376665+0000","last_peered":"2023-10-07T23:15:31.376665+0000","last_clean":"2023-10-07T23:15:31.376665+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:15:31.376665+0000","last_undegraded":"2023-10-07T23:15:31.376665+0000","last_fullsized":"2023-10-07T23:15:31.376665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.375387+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:15:31.375387+0000","last_peered":"2023-10-07T23:15:31.375387+0000","last_clean":"2023-10-07T23:15:31.375387+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:15:31.375387+0000","last_undegraded":"2023-10-07T23:15:31.375387+0000","last_fullsized":"2023-10-07T23:15:31.375387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.372592+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:15:31.372592+0000","last_peered":"2023-10-07T23:15:31.372592+0000","last_clean":"2023-10-07T23:15:31.372592+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:15:31.372592+0000","last_undegraded":"2023-10-07T23:15:31.372592+0000","last_fullsized":"2023-10-07T23:15:31.372592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.369687+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:15:31.369687+0000","last_peered":"2023-10-07T23:15:31.369687+0000","last_clean":"2023-10-07T23:15:31.369687+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:15:31.369687+0000","last_undegraded":"2023-10-07T23:15:31.369687+0000","last_fullsized":"2023-10-07T23:15:31.369687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-07T23:15:31.365104+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:15:31.365104+0000","last_peered":"2023-10-07T23:15:31.365104+0000","last_clean":"2023-10-07T23:15:31.365104+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:15:31.365104+0000","last_undegraded":"2023-10-07T23:15:31.365104+0000","last_fullsized":"2023-10-07T23:15:31.365104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6017,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":17107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383695,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446908,"statfs":{"total":95995035648,"available":95689633792,"internally_reserved":0,"allocated":978944,"data_stored":475491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.51200000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.79400000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.75900000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298856,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446296,"statfs":{"total":95995035648,"available":95689007104,"internally_reserved":0,"allocated":1605632,"data_stored":1070179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35599999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58899999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.504}]},{"osd":5,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56000000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.57899999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.73799999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298260,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446892,"statfs":{"total":95995035648,"available":95689617408,"internally_reserved":0,"allocated":995328,"data_stored":476247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68200000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.88,"5min":1.054,"15min":1.054},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.627}]},{"osd":2,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.84699999999999998,"15min":0.86099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.325,"15min":1.325},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":1.278,"15min":1.278},"last":0.79900000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.67600000000000005,"15min":0.69799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76700000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":1.385,"15min":1.385},"last":0.60899999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.52800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97799999999999998,"5min":1.345,"15min":1.345},"last":0.752},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.84799999999999998,"15min":0.873},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.82299999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.90100000000000002,"15min":0.92700000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97599999999999998,"5min":1.579,"15min":1.579},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.96399999999999997,"15min":0.99299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0649999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.113,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.018,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2310000000000001,"5min":1.603,"15min":1.603},"last":0.74099999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.026,"15min":1.056},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.038,"5min":1.782,"15min":1.782},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.96599999999999997,"15min":0.998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0529999999999999,"5min":1.956,"15min":1.956},"last":0.78600000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298256,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446896,"statfs":{"total":95995035648,"available":95689621504,"internally_reserved":0,"allocated":991232,"data_stored":476089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.755,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76300000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.66200000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.72799999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63900000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.498},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75600000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85299999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56299999999999994}]},{"osd":4,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.746}]},{"osd":5,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82699999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84299999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96799999999999997,"5min":1.597,"15min":1.597},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.872,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88400000000000001,"5min":1,"15min":1},"last":0.82599999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.59899999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92900000000000005,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96399999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Sat Oct 7 23:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.91600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298828,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446324,"statfs":{"total":95995035648,"available":95689035776,"internally_reserved":0,"allocated":1576960,"data_stored":1066413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.502},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.44600000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44600000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69099999999999995,"15min":0.68000000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.54000000000000004}]},{"osd":8,"last update":"Sat Oct 7 23:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98599999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68000000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:14:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":475168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70999999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70999999999999996,"5min":0.878,"15min":0.878},"last":0.64700000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.70999999999999996,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.5700000000000003,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.56299999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68899999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.877,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.95599999999999996,"5min":1.014,"15min":1.014},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75,"15min":0.751},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.099,"15min":1.099},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.127,"5min":0.87,"15min":0.82699999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":6.524,"5min":6.524,"15min":6.524},"last":0.84599999999999997}]},{"osd":8,"last update":"Sat Oct 7 23:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85799999999999998,"15min":0.85599999999999998},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.0509999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88800000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0580000000000001,"5min":1.165,"15min":1.165},"last":0.81000000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:15:15 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":6.5890000000000004,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446900,"statfs":{"total":95995035648,"available":95689625600,"internally_reserved":0,"allocated":987136,"data_stored":478883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.624},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78700000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92800000000000005,"5min":1.014,"15min":1.014},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.42599999999999999},"min":{"1min":0.28499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.501}]},{"osd":5,"last update":"Sat Oct 7 23:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.56599999999999995,"15min":0.53400000000000003},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72599999999999998}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.752}]},{"osd":10,"last update":"Sat Oct 7 23:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446900,"statfs":{"total":95995035648,"available":95689625600,"internally_reserved":0,"allocated":987136,"data_stored":475834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.874},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":1.022,"15min":1.022},"last":0.79200000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.45500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.40200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69699999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.80800000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.89100000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.52100000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":1.155,"15min":1.155},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.83199999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78900000000000003,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95499999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.93700000000000006,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.91900000000000004}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2150000000000001,"5min":1.323,"15min":1.323},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.97299999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.159},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.99299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298256,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446896,"statfs":{"total":95995035648,"available":95689621504,"internally_reserved":0,"allocated":991232,"data_stored":478917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52900000000000003,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.747,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.502},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59099999999999997,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.58199999999999996},"min":{"1min":0.47199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83399999999999996,"5min":0.871,"15min":0.871},"last":0.80200000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.752}]},{"osd":5,"last update":"Sat Oct 7 23:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.439,"15min":0.433},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65100000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.46200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76500000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.436,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76800000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68400000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81999999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84699999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.85799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446900,"statfs":{"total":95995035648,"available":95689625600,"internally_reserved":0,"allocated":987136,"data_stored":475595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.32300000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.436},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.875,"5min":0.999,"15min":0.999},"last":0.60999999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.59899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.0109999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.75},{"interface":"front","average":{"1min":0.625,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.60599999999999998},"min":{"1min":0.53400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.65300000000000002,"15min":0.60899999999999999},"min":{"1min":0.57699999999999996,"5min":0.434,"15min":0.434},"max":{"1min":0.98999999999999999,"5min":1.218,"15min":1.218},"last":0.84699999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.80400000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68700000000000006,"15min":0.67400000000000004},"min":{"1min":0.498,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Oct 7 23:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.374,"15min":0.374},"max":{"1min":1.048,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.996,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.94999999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.71399999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2230000000000001,"5min":1.288,"15min":1.288},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0029999999999999,"15min":1.0189999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.169,"5min":1.51,"15min":1.51},"last":0.97299999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]}]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446904,"statfs":{"total":95995035648,"available":95689629696,"internally_reserved":0,"allocated":983040,"data_stored":475513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.36499999999999999},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.372},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.38800000000000001,"15min":0.35299999999999998},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.49399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.49399999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.46999999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.53800000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56999999999999995,"15min":0.54200000000000004},"min":{"1min":0.51800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.55500000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.63700000000000001},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.73299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77300000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78000000000000003}]},{"osd":8,"last update":"Sat Oct 7 23:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.80200000000000005,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63500000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298848,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446304,"statfs":{"total":95995035648,"available":95689015296,"internally_reserved":0,"allocated":1597440,"data_stored":1066936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.499,"15min":0.47599999999999998},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.502,"15min":0.47999999999999998},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46300000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.38300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.505,"15min":0.48899999999999999},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.374,"15min":0.35499999999999998},"min":{"1min":0.25,"5min":0.217,"15min":0.217},"max":{"1min":0.59399999999999997,"5min":0.629,"15min":0.629},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.371},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.438}]},{"osd":3,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.55600000000000005,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Sat Oct 7 23:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64000000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80100000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Oct 7 23:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.78000000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:15:55.707 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T23:15:55.707 INFO:tasks.cephadm:Setting up client nodes... 2023-10-07T23:15:55.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-07T23:15:56.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:56 smithi123 ceph-mon[122926]: from='client.24743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:15:56.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:56 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] up:active 2023-10-07T23:15:56.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:56 smithi123 ceph-mon[122926]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-10-07T23:15:56.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:56 smithi123 ceph-mon[122926]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-07T23:15:56.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:56 smithi086 ceph-mon[122131]: from='client.24743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:15:56.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:56 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] up:active 2023-10-07T23:15:56.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:56 smithi086 ceph-mon[122131]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-10-07T23:15:56.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:56 smithi086 ceph-mon[122131]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-07T23:15:56.448 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:55 smithi086 conmon[166399]: 2023-10-07T23:15:55.981+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-07T23:15:56.448 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:55 smithi086 conmon[166399]: 2023-10-07T23:15:55.981+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-10-07T23:15:56.448 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:15:55 smithi086 conmon[166399]: 2023-10-07T23:15:55.981+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-10-07T23:15:56.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:56 smithi067 ceph-mon[82723]: from='client.24743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:15:56.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:56 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] up:active 2023-10-07T23:15:56.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:56 smithi067 ceph-mon[82723]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-10-07T23:15:56.505 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:56 smithi067 ceph-mon[82723]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-07T23:15:56.506 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:15:57.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:57 smithi067 ceph-mon[82723]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:15:57.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:57 smithi067 ceph-mon[82723]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.8 KiB/s wr, 5 op/s 2023-10-07T23:15:57.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:57 smithi123 ceph-mon[122926]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:15:57.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:57 smithi123 ceph-mon[122926]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.8 KiB/s wr, 5 op/s 2023-10-07T23:15:57.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:57 smithi086 ceph-mon[122131]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:15:57.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:57 smithi086 ceph-mon[122131]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.8 KiB/s wr, 5 op/s 2023-10-07T23:15:58.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.021+0000 7f618e983700 1 -- 172.21.15.67:0/3382596679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 msgr2=0x7f6188107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:58.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.021+0000 7f618e983700 1 --2- 172.21.15.67:0/3382596679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 0x7f6188107180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6180004ca0 tx=0x7f6180009ca0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 -- 172.21.15.67:0/3382596679 shutdown_connections 2023-10-07T23:15:58.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 --2- 172.21.15.67:0/3382596679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6188071bc0 0x7f6188070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 --2- 172.21.15.67:0/3382596679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 0x7f6188107180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 --2- 172.21.15.67:0/3382596679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188070570 0x7f61880709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 -- 172.21.15.67:0/3382596679 >> 172.21.15.67:0/3382596679 conn(0x7f61880f85c0 msgr2=0x7f61880fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:58.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 -- 172.21.15.67:0/3382596679 shutdown_connections 2023-10-07T23:15:58.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 -- 172.21.15.67:0/3382596679 wait complete. 2023-10-07T23:15:58.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.022+0000 7f618e983700 1 Processor -- start 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 -- start start 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6188070570 0x7f618810fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 0x7f618810fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 0x7f6188117080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188117690 con 0x7f6188071bc0 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188117800 con 0x7f6188070570 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618e983700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881179c0 con 0x7f6188104d00 2023-10-07T23:15:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f61877fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 0x7f618810fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f6187fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6188070570 0x7f618810fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f61877fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 0x7f618810fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56072/0 (socket says 172.21.15.67:56072) 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f61877fe700 1 -- 172.21.15.67:0/959499581 learned_addr learned my addr 172.21.15.67:0/959499581 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.023+0000 7f618cf20700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 0x7f6188117080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f61877fe700 1 -- 172.21.15.67:0/959499581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 msgr2=0x7f6188117080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f61877fe700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 0x7f6188117080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f61877fe700 1 -- 172.21.15.67:0/959499581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6188070570 msgr2=0x7f618810fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f61877fe700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6188070570 0x7f618810fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f61877fe700 1 -- 172.21.15.67:0/959499581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c005040 con 0x7f6188071bc0 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f61877fe700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 0x7f618810fff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f617c009730 tx=0x7f617c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c007be0 con 0x7f6188071bc0 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617c00f040 con 0x7f6188071bc0 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c013bc0 con 0x7f6188071bc0 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.024+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61800048f0 con 0x7f6188071bc0 2023-10-07T23:15:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.025+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6188117fd0 con 0x7f6188071bc0 2023-10-07T23:15:58.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.025+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6188103b60 con 0x7f6188071bc0 2023-10-07T23:15:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.027+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f617c00e840 con 0x7f6188071bc0 2023-10-07T23:15:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.029+0000 7f6184ff9700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f617009b7f0 0x7f617009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:15:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.029+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f617c0c4f30 con 0x7f6188071bc0 2023-10-07T23:15:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.029+0000 7f6187fff700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f617009b7f0 0x7f617009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:15:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.029+0000 7f6187fff700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f617009b7f0 0x7f617009dca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6178009b80 tx=0x7f617800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:15:58.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.032+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f617c090500 con 0x7f6188071bc0 2023-10-07T23:15:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.314+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f618810a850 con 0x7f6188071bc0 2023-10-07T23:15:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.319+0000 7f6184ff9700 1 -- 172.21.15.67:0/959499581 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f617c093b20 con 0x7f6188071bc0 2023-10-07T23:15:58.320 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2023-10-07T23:15:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCu5iFlBKbaEhAAUDYgzkvFR1ZPd7mXJe/R8g== 2023-10-07T23:15:58.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f617009b7f0 msgr2=0x7f617009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f617009b7f0 0x7f617009dca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6178009b80 tx=0x7f617800b040 comp rx=0 tx=0).stop 2023-10-07T23:15:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 msgr2=0x7f618810fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:15:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 0x7f618810fff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f617c009730 tx=0x7f617c009e30 comp rx=0 tx=0).stop 2023-10-07T23:15:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 shutdown_connections 2023-10-07T23:15:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6188070570 0x7f618810fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.321+0000 7f618e983700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f617009b7f0 0x7f617009dca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.322+0000 7f618e983700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6188104d00 0x7f6188117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.322+0000 7f618e983700 1 --2- 172.21.15.67:0/959499581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6188071bc0 0x7f618810fff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:15:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.322+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 >> 172.21.15.67:0/959499581 conn(0x7f61880f85c0 msgr2=0x7f6188100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:15:58.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.322+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 shutdown_connections 2023-10-07T23:15:58.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:15:58.322+0000 7f618e983700 1 -- 172.21.15.67:0/959499581 wait complete. 2023-10-07T23:15:58.794 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-07T23:15:58.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-07T23:15:58.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-07T23:15:58.890 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-07T23:15:58.890 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-07T23:15:58.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-07T23:15:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:59 smithi067 ceph-mon[82723]: pgmap v304: 129 pgs: 129 active+clean; 585 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-07T23:15:59.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:59 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/959499581' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-07T23:15:59.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:15:59 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/959499581' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-07T23:15:59.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:59 smithi123 ceph-mon[122926]: pgmap v304: 129 pgs: 129 active+clean; 585 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-07T23:15:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:59 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/959499581' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-07T23:15:59.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:15:59 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/959499581' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-07T23:15:59.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:59 smithi086 ceph-mon[122131]: pgmap v304: 129 pgs: 129 active+clean; 585 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-10-07T23:15:59.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:59 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/959499581' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-07T23:15:59.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:15:59 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/959499581' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-07T23:15:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.311+0000 7ff486bd7700 1 -- 172.21.15.67:0/843731461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 msgr2=0x7ff4800fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.311+0000 7ff486bd7700 1 --2- 172.21.15.67:0/843731461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff4800fce70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4700049f0 tx=0x7ff470009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 -- 172.21.15.67:0/843731461 shutdown_connections 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 --2- 172.21.15.67:0/843731461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 --2- 172.21.15.67:0/843731461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 0x7ff480103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 --2- 172.21.15.67:0/843731461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff4800fce70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 -- 172.21.15.67:0/843731461 >> 172.21.15.67:0/843731461 conn(0x7ff480074320 msgr2=0x7ff480074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 -- 172.21.15.67:0/843731461 shutdown_connections 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 -- 172.21.15.67:0/843731461 wait complete. 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.312+0000 7ff486bd7700 1 Processor -- start 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 -- start start 2023-10-07T23:16:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 0x7ff480116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff48010fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4801103e0 con 0x7ff480103800 2023-10-07T23:16:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff480110550 con 0x7ff4800fd550 2023-10-07T23:16:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff486bd7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff480110710 con 0x7ff480102e60 2023-10-07T23:16:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff484973700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff484973700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:43124/0 (socket says 172.21.15.67:43124) 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff484973700 1 -- 172.21.15.67:0/1362671172 learned_addr learned my addr 172.21.15.67:0/1362671172 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff485174700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff48010fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.313+0000 7ff477fff700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 0x7ff480116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff484973700 1 -- 172.21.15.67:0/1362671172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 msgr2=0x7ff480116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff484973700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 0x7ff480116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff484973700 1 -- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 msgr2=0x7ff48010fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff484973700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff48010fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff484973700 1 -- 172.21.15.67:0/1362671172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470005040 con 0x7ff4800fd550 2023-10-07T23:16:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff477fff700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 0x7ff480116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff485174700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff48010fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff484973700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480115d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00ebf0 tx=0x7ff47c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c007cc0 con 0x7ff4800fd550 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff480110990 con 0x7ff4800fd550 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff47c00f040 con 0x7ff4800fd550 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0094c0 con 0x7ff4800fd550 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.314+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48011cd00 con 0x7ff4800fd550 2023-10-07T23:16:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.315+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480063f20 con 0x7ff4800fd550 2023-10-07T23:16:01.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.317+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7ff47c00c680 con 0x7ff4800fd550 2023-10-07T23:16:01.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.319+0000 7ff4757fa700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff46809b770 0x7ff46809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:01.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.319+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7ff47c0c5270 con 0x7ff4800fd550 2023-10-07T23:16:01.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.319+0000 7ff477fff700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff46809b770 0x7ff46809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:01.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.319+0000 7ff477fff700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff46809b770 0x7ff46809dc20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4801115b0 tx=0x7ff470009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:01.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.320+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7ff47c0908a0 con 0x7ff4800fd550 2023-10-07T23:16:01.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:01 smithi123 ceph-mon[122926]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2023-10-07T23:16:01.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:01 smithi086 ceph-mon[122131]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2023-10-07T23:16:01.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:01 smithi067 ceph-mon[82723]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2023-10-07T23:16:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.580+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff48010cc80 con 0x7ff4800fd550 2023-10-07T23:16:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.586+0000 7ff4757fa700 1 -- 172.21.15.67:0/1362671172 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+249411 (secure 0 0 0) 0x7ff47c019020 con 0x7ff4800fd550 2023-10-07T23:16:01.591 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:01.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff46809b770 msgr2=0x7ff46809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff46809b770 0x7ff46809dc20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4801115b0 tx=0x7ff470009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 msgr2=0x7ff480115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480115d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00ebf0 tx=0x7ff47c00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 shutdown_connections 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4800fd550 0x7ff480115d10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff46809b770 0x7ff46809dc20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff480103800 0x7ff48010fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 --2- 172.21.15.67:0/1362671172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff480102e60 0x7ff480116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 >> 172.21.15.67:0/1362671172 conn(0x7ff480074320 msgr2=0x7ff48006fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 shutdown_connections 2023-10-07T23:16:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:01.591+0000 7ff486bd7700 1 -- 172.21.15.67:0/1362671172 wait complete. 2023-10-07T23:16:01.981 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2299203628},{"type":"v1","addr":"172.21.15.67:6801","nonce":2299203628}]},"active_addr":"172.21.15.67:6801/2299203628","active_change":"2023-10-07T23:08:27.835018+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2999755584}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2002734168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1459014596}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3887664352}]}]} 2023-10-07T23:16:01.987 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-07T23:16:01.988 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-07T23:16:01.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-07T23:16:02.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:02 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1362671172' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-07T23:16:02.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:02 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1362671172' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-07T23:16:02.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:02 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1362671172' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-07T23:16:02.763 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:03.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:03 smithi067 ceph-mon[82723]: pgmap v306: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2023-10-07T23:16:03.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:03 smithi123 ceph-mon[122926]: pgmap v306: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2023-10-07T23:16:03.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:03 smithi086 ceph-mon[122131]: pgmap v306: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2023-10-07T23:16:04.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 -- 172.21.15.67:0/1597918665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 msgr2=0x7fc474105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:04.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 --2- 172.21.15.67:0/1597918665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474105440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc464004b40 tx=0x7fc464009dd0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 -- 172.21.15.67:0/1597918665 shutdown_connections 2023-10-07T23:16:04.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 --2- 172.21.15.67:0/1597918665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4740ff760 0x7fc47410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 --2- 172.21.15.67:0/1597918665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474105440 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 --2- 172.21.15.67:0/1597918665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474105a10 0x7fc4740ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 -- 172.21.15.67:0/1597918665 >> 172.21.15.67:0/1597918665 conn(0x7fc4740fabc0 msgr2=0x7fc4740fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:04.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 -- 172.21.15.67:0/1597918665 shutdown_connections 2023-10-07T23:16:04.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.270+0000 7fc478c63700 1 -- 172.21.15.67:0/1597918665 wait complete. 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 Processor -- start 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 -- start start 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4740ff760 0x7fc474115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474105a10 0x7fc47410fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4741103f0 con 0x7fc474105070 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474110560 con 0x7fc474105a10 2023-10-07T23:16:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.271+0000 7fc478c63700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474110720 con 0x7fc4740ff760 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42210/0 (socket says 172.21.15.67:42210) 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 -- 172.21.15.67:0/804667559 learned_addr learned my addr 172.21.15.67:0/804667559 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc47259c700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4740ff760 0x7fc474115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc472d9d700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474105a10 0x7fc47410fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 -- 172.21.15.67:0/804667559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4740ff760 msgr2=0x7fc474115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4740ff760 0x7fc474115d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 -- 172.21.15.67:0/804667559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474105a10 msgr2=0x7fc47410fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474105a10 0x7fc47410fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 -- 172.21.15.67:0/804667559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464004650 con 0x7fc474105070 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.272+0000 7fc471d9b700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474116260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc46800ebc0 tx=0x7fc46800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:04.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.273+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468007cc0 con 0x7fc474105070 2023-10-07T23:16:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.273+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46800f040 con 0x7fc474105070 2023-10-07T23:16:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.273+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4741109a0 con 0x7fc474105070 2023-10-07T23:16:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.273+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800c990 con 0x7fc474105070 2023-10-07T23:16:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.273+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47411cd10 con 0x7fc474105070 2023-10-07T23:16:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.274+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc474063f20 con 0x7fc474105070 2023-10-07T23:16:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.276+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fc46800cd40 con 0x7fc474105070 2023-10-07T23:16:04.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.277+0000 7fc462ffd700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc45c09b7f0 0x7fc45c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:04.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.277+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fc4680c4af0 con 0x7fc474105070 2023-10-07T23:16:04.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.278+0000 7fc47259c700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc45c09b7f0 0x7fc45c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:04.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.278+0000 7fc47259c700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc45c09b7f0 0x7fc45c09dca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc464009ec0 tx=0x7fc464016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.281+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fc46804da00 con 0x7fc474105070 2023-10-07T23:16:04.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.485+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc47410ee90 con 0x7fc474105070 2023-10-07T23:16:04.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.486+0000 7fc462ffd700 1 -- 172.21.15.67:0/804667559 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7fc468019020 con 0x7fc474105070 2023-10-07T23:16:04.487 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:04.487 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc45c09b7f0 msgr2=0x7fc45c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc45c09b7f0 0x7fc45c09dca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc464009ec0 tx=0x7fc464016040 comp rx=0 tx=0).stop 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 msgr2=0x7fc474116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474116260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc46800ebc0 tx=0x7fc46800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 shutdown_connections 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc474105a10 0x7fc47410fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc45c09b7f0 0x7fc45c09dca0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4740ff760 0x7fc474115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 --2- 172.21.15.67:0/804667559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc474105070 0x7fc474116260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:04.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 >> 172.21.15.67:0/804667559 conn(0x7fc4740fabc0 msgr2=0x7fc4740fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:04.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 shutdown_connections 2023-10-07T23:16:04.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:04.488+0000 7fc478c63700 1 -- 172.21.15.67:0/804667559 wait complete. 2023-10-07T23:16:04.948 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-07T23:16:04.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-07T23:16:05.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:04 smithi123 ceph-mon[122926]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 22 op/s 2023-10-07T23:16:05.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:04 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/804667559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:04 smithi086 ceph-mon[122131]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 22 op/s 2023-10-07T23:16:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:04 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/804667559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:04 smithi067 ceph-mon[82723]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 22 op/s 2023-10-07T23:16:05.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:04 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/804667559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:05.721 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 -- 172.21.15.67:0/566305890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 msgr2=0x7f3fbc0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/566305890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc0fce70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3fac004930 tx=0x7f3fac009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 -- 172.21.15.67:0/566305890 shutdown_connections 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/566305890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 0x7f3fbc103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/566305890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fbc0fd550 0x7f3fbc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/566305890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc0fce70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.181+0000 7f3fc46cc700 1 -- 172.21.15.67:0/566305890 >> 172.21.15.67:0/566305890 conn(0x7f3fbc074320 msgr2=0x7f3fbc074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.182+0000 7f3fc46cc700 1 -- 172.21.15.67:0/566305890 shutdown_connections 2023-10-07T23:16:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.182+0000 7f3fc46cc700 1 -- 172.21.15.67:0/566305890 wait complete. 2023-10-07T23:16:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.182+0000 7f3fc46cc700 1 Processor -- start 2023-10-07T23:16:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.182+0000 7f3fc46cc700 1 -- start start 2023-10-07T23:16:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc46cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fbc0fd550 0x7f3fbc115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc46cc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 0x7f3fbc116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc46cc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc10fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc46cc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc110320 con 0x7f3fbc103800 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc46cc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc110490 con 0x7f3fbc102e60 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc46cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc110650 con 0x7f3fbc0fd550 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc2c69700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc10fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc2c69700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc10fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42224/0 (socket says 172.21.15.67:42224) 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc2c69700 1 -- 172.21.15.67:0/1923974712 learned_addr learned my addr 172.21.15.67:0/1923974712 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc2468700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fbc0fd550 0x7f3fbc115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.183+0000 7f3fc1c67700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 0x7f3fbc116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc2c69700 1 -- 172.21.15.67:0/1923974712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fbc0fd550 msgr2=0x7f3fbc115c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc2c69700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fbc0fd550 0x7f3fbc115c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc2c69700 1 -- 172.21.15.67:0/1923974712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 msgr2=0x7f3fbc116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc2c69700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 0x7f3fbc116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc2c69700 1 -- 172.21.15.67:0/1923974712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac005040 con 0x7f3fbc103800 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc1c67700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 0x7f3fbc116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc2c69700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc10fd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400eb70 tx=0x7f3fb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40047d0 con 0x7f3fbc103800 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1108d0 con 0x7f3fbc103800 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb400f040 con 0x7f3fbc103800 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400c820 con 0x7f3fbc103800 2023-10-07T23:16:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.184+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc11cc40 con 0x7f3fbc103800 2023-10-07T23:16:07.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.185+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc063f20 con 0x7f3fbc103800 2023-10-07T23:16:07.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.187+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f3fb400cc50 con 0x7f3fbc103800 2023-10-07T23:16:07.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.188+0000 7f3fb2ffd700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3fa809b770 0x7f3fa809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:07.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.189+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f3fb40c4e00 con 0x7f3fbc103800 2023-10-07T23:16:07.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.189+0000 7f3fc2468700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3fa809b770 0x7f3fa809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:07.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.189+0000 7f3fc2468700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3fa809b770 0x7f3fa809dc20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8001470 tx=0x7f3fb800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:07.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.192+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f3fb40903d0 con 0x7f3fbc103800 2023-10-07T23:16:07.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:06 smithi067 ceph-mon[82723]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 18 op/s 2023-10-07T23:16:07.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:06 smithi123 ceph-mon[122926]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 18 op/s 2023-10-07T23:16:07.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.400+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fbc10cc60 con 0x7f3fbc103800 2023-10-07T23:16:07.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.401+0000 7f3fb2ffd700 1 -- 172.21.15.67:0/1923974712 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23167 (secure 0 0 0) 0x7f3fb4019020 con 0x7f3fbc103800 2023-10-07T23:16:07.402 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:07.402 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:15:31.863092+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:16:07.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3fa809b770 msgr2=0x7f3fa809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3fa809b770 0x7f3fa809dc20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8001470 tx=0x7f3fb800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 msgr2=0x7f3fbc10fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc10fd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400eb70 tx=0x7f3fb400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 shutdown_connections 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3fa809b770 0x7f3fa809dc20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fbc103800 0x7f3fbc10fd00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fbc0fd550 0x7f3fbc115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 --2- 172.21.15.67:0/1923974712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102e60 0x7f3fbc116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 >> 172.21.15.67:0/1923974712 conn(0x7f3fbc074320 msgr2=0x7f3fbc06fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 shutdown_connections 2023-10-07T23:16:07.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:07.404+0000 7f3fc46cc700 1 -- 172.21.15.67:0/1923974712 wait complete. 2023-10-07T23:16:07.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:06 smithi086 ceph-mon[122131]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 18 op/s 2023-10-07T23:16:07.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-07T23:16:07.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-07T23:16:07.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-07T23:16:07.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-07T23:16:07.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-07T23:16:07.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-07T23:16:07.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-07T23:16:07.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-07T23:16:07.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-07T23:16:07.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-07T23:16:07.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-07T23:16:07.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-07T23:16:08.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:07 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1923974712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:08.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:07 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1923974712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:08.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:07 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1923974712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:09.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:08 smithi067 ceph-mon[82723]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 20 op/s 2023-10-07T23:16:09.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:08 smithi123 ceph-mon[122926]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 20 op/s 2023-10-07T23:16:09.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:08 smithi086 ceph-mon[122131]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 20 op/s 2023-10-07T23:16:09.764 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:09.977 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.030 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.047 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.164 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.191 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.353 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:10.441 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:11.111 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:10 smithi067 ceph-mon[82723]: pgmap v310: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-10-07T23:16:11.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:10 smithi123 ceph-mon[122926]: pgmap v310: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-10-07T23:16:11.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:10 smithi086 ceph-mon[122131]: pgmap v310: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-10-07T23:16:11.522 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:12.214 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:13.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:13 smithi067 ceph-mon[82723]: pgmap v311: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-07T23:16:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:13 smithi123 ceph-mon[122926]: pgmap v311: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-07T23:16:13.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:13 smithi086 ceph-mon[122131]: pgmap v311: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-07T23:16:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 -- 172.21.15.67:0/16195300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af4103800 msgr2=0x7f6af40fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 --2- 172.21.15.67:0/16195300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af4103800 0x7f6af40fce70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80049f0 tx=0x7f6ae8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 -- 172.21.15.67:0/16195300 shutdown_connections 2023-10-07T23:16:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 --2- 172.21.15.67:0/16195300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af40fd550 0x7f6af4109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 --2- 172.21.15.67:0/16195300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af4102e60 0x7f6af4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:14.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 --2- 172.21.15.67:0/16195300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af4103800 0x7f6af40fce70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 -- 172.21.15.67:0/16195300 >> 172.21.15.67:0/16195300 conn(0x7f6af4074320 msgr2=0x7f6af4074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 -- 172.21.15.67:0/16195300 shutdown_connections 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.203+0000 7f6afa73e700 1 -- 172.21.15.67:0/16195300 wait complete. 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 Processor -- start 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 -- start start 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af40fd550 0x7f6af4115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af4102e60 0x7f6af41162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af4103800 0x7f6af410fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4110440 con 0x7f6af40fd550 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af41105b0 con 0x7f6af4103800 2023-10-07T23:16:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.204+0000 7f6afa73e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4110770 con 0x7f6af4102e60 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af40fd550 0x7f6af4115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af40fd550 0x7f6af4115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57640/0 (socket says 172.21.15.67:57640) 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 -- 172.21.15.67:0/207272147 learned_addr learned my addr 172.21.15.67:0/207272147 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af37fe700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af4102e60 0x7f6af41162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af8cdb700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af4103800 0x7f6af410fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 -- 172.21.15.67:0/207272147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af4102e60 msgr2=0x7f6af41162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af4102e60 0x7f6af41162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 -- 172.21.15.67:0/207272147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af4103800 msgr2=0x7f6af410fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af4103800 0x7f6af410fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 -- 172.21.15.67:0/207272147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8005040 con 0x7f6af40fd550 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af37fe700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af4102e60 0x7f6af41162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:14.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.205+0000 7f6af3fff700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af40fd550 0x7f6af4115d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400ebc0 tx=0x7f6ae400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.206+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4007cc0 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.206+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af41109f0 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.206+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae400f040 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.206+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40094a0 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.206+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af411cd60 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.207+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6af4109ab0 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.209+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f6ae400c600 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af0ff9700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6adc09b770 0x7f6adc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f6ae40c4d40 con 0x7f6af40fd550 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af37fe700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6adc09b770 0x7f6adc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:14.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af0ff9700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] conn(0x7f6adc0a62e0 0x7f6adc0a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 --> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6adc0a8de0 con 0x7f6adc0a62e0 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae40c5030 con 0x7f6af40fd550 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.211+0000 7f6af8cdb700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] conn(0x7f6adc0a62e0 0x7f6adc0a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.212+0000 7f6af37fe700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6adc09b770 0x7f6adc09dc20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800b090 tx=0x7f6ae8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.212+0000 7f6af8cdb700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] conn(0x7f6adc0a62e0 0x7f6adc0a8710 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.213+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== osd.0 v2:172.21.15.67:6802/1635424780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6adc0a8de0 con 0x7f6adc0a62e0 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.229+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 --> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6af4063f20 con 0x7f6adc0a62e0 2023-10-07T23:16:14.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.229+0000 7f6af0ff9700 1 -- 172.21.15.67:0/207272147 <== osd.0 v2:172.21.15.67:6802/1635424780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6af4063f20 con 0x7f6adc0a62e0 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] conn(0x7f6adc0a62e0 msgr2=0x7f6adc0a8710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] conn(0x7f6adc0a62e0 0x7f6adc0a8710 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6adc09b770 msgr2=0x7f6adc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6adc09b770 0x7f6adc09dc20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800b090 tx=0x7f6ae8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af40fd550 msgr2=0x7f6af4115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 --2- 172.21.15.67:0/207272147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6af40fd550 0x7f6af4115d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400ebc0 tx=0x7f6ae400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6af8cdb700 1 -- 172.21.15.67:0/207272147 reap_dead start 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 shutdown_connections 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 >> 172.21.15.67:0/207272147 conn(0x7f6af4074320 msgr2=0x7f6af4108050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 shutdown_connections 2023-10-07T23:16:14.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:14.230+0000 7f6afa73e700 1 -- 172.21.15.67:0/207272147 wait complete. 2023-10-07T23:16:15.099 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:14 smithi067 ceph-mon[82723]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-07T23:16:15.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:14 smithi123 ceph-mon[122926]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-07T23:16:15.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:14 smithi086 ceph-mon[122131]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-07T23:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 -- 172.21.15.67:0/684871545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314103800 msgr2=0x7f13140fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 --2- 172.21.15.67:0/684871545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314103800 0x7f13140fce70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13040049f0 tx=0x7f1304009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 -- 172.21.15.67:0/684871545 shutdown_connections 2023-10-07T23:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 --2- 172.21.15.67:0/684871545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314102e60 0x7f1314103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 --2- 172.21.15.67:0/684871545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13140fd550 0x7f1314109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 --2- 172.21.15.67:0/684871545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314103800 0x7f13140fce70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 -- 172.21.15.67:0/684871545 >> 172.21.15.67:0/684871545 conn(0x7f1314074320 msgr2=0x7f1314074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 -- 172.21.15.67:0/684871545 shutdown_connections 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.456+0000 7f131c5e3700 1 -- 172.21.15.67:0/684871545 wait complete. 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.457+0000 7f131c5e3700 1 Processor -- start 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.457+0000 7f131c5e3700 1 -- start start 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.457+0000 7f131c5e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13140fd550 0x7f1314113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.457+0000 7f131c5e3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314102e60 0x7f1314113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.457+0000 7f131c5e3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314103800 0x7f131410dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f131c5e3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131410e2f0 con 0x7f1314102e60 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f131c5e3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131410e460 con 0x7f1314103800 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f131c5e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131410e620 con 0x7f13140fd550 2023-10-07T23:16:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314102e60 0x7f1314113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f131a37f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13140fd550 0x7f1314113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314102e60 0x7f1314113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57650/0 (socket says 172.21.15.67:57650) 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 -- 172.21.15.67:0/1286484473 learned_addr learned my addr 172.21.15.67:0/1286484473 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f131ab80700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314103800 0x7f131410dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 -- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13140fd550 msgr2=0x7f1314113a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13140fd550 0x7f1314113a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 -- 172.21.15.67:0/1286484473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314103800 msgr2=0x7f131410dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1314103800 0x7f131410dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.458+0000 7f1319b7e700 1 -- 172.21.15.67:0/1286484473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304005040 con 0x7f1314102e60 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.459+0000 7f1319b7e700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314102e60 0x7f1314113f50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f130400c010 tx=0x7f1304009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.459+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400e040 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.459+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13040095f0 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.459+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131410e8a0 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.459+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304013e30 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.459+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1314070030 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.460+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13140fbe00 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.462+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f1304013420 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.463+0000 7f130affd700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130009b7f0 0x7f130009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.463+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f13040c6f60 con 0x7f1314102e60 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.464+0000 7f131a37f700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130009b7f0 0x7f130009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.464+0000 7f130affd700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] conn(0x7f13000a6360 0x7f13000a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.464+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 --> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13000a8e60 con 0x7f13000a6360 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.464+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13040c7250 con 0x7f1314102e60 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.464+0000 7f131a37f700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130009b7f0 0x7f130009dca0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1310009b80 tx=0x7f131000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.464+0000 7f131ab80700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] conn(0x7f13000a6360 0x7f13000a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.465+0000 7f131ab80700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] conn(0x7f13000a6360 0x7f13000a8790 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.466+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== osd.2 v2:172.21.15.123:6800/3773700997 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f13000a8e60 con 0x7f13000a6360 2023-10-07T23:16:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.484+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 --> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1314063f20 con 0x7f13000a6360 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f130affd700 1 -- 172.21.15.67:0/1286484473 <== osd.2 v2:172.21.15.123:6800/3773700997 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1314063f20 con 0x7f13000a6360 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] conn(0x7f13000a6360 msgr2=0x7f13000a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131c5e3700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] conn(0x7f13000a6360 0x7f13000a8790 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130009b7f0 msgr2=0x7f130009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131c5e3700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f130009b7f0 0x7f130009dca0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1310009b80 tx=0x7f131000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314102e60 msgr2=0x7f1314113f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131c5e3700 1 --2- 172.21.15.67:0/1286484473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1314102e60 0x7f1314113f50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f130400c010 tx=0x7f1304009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.485+0000 7f131ab80700 1 -- 172.21.15.67:0/1286484473 reap_dead start 2023-10-07T23:16:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.486+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 shutdown_connections 2023-10-07T23:16:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.486+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 >> 172.21.15.67:0/1286484473 conn(0x7f1314074320 msgr2=0x7f13140fb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.486+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 shutdown_connections 2023-10-07T23:16:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.486+0000 7f131c5e3700 1 -- 172.21.15.67:0/1286484473 wait complete. 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 -- 172.21.15.67:0/875999485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 msgr2=0x7f97701091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 --2- 172.21.15.67:0/875999485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 0x7f97701091d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97640049c0 tx=0x7f9764009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 -- 172.21.15.67:0/875999485 shutdown_connections 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 --2- 172.21.15.67:0/875999485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 0x7f9770070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 --2- 172.21.15.67:0/875999485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770109710 0x7f977010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 --2- 172.21.15.67:0/875999485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 0x7f97701091d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 -- 172.21.15.67:0/875999485 >> 172.21.15.67:0/875999485 conn(0x7f9770074320 msgr2=0x7f9770074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 -- 172.21.15.67:0/875999485 shutdown_connections 2023-10-07T23:16:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.897+0000 7f97762d2700 1 -- 172.21.15.67:0/875999485 wait complete. 2023-10-07T23:16:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.898+0000 7f97762d2700 1 Processor -- start 2023-10-07T23:16:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 -- start start 2023-10-07T23:16:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 0x7f9770115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 0x7f9770116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770109710 0x7f977010fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9770110400 con 0x7f9770070890 2023-10-07T23:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9770110570 con 0x7f977006ff80 2023-10-07T23:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.899+0000 7f97762d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9770110730 con 0x7f9770109710 2023-10-07T23:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 0x7f9770115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 0x7f9770115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:42734/0 (socket says 172.21.15.67:42734) 2023-10-07T23:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 -- 172.21.15.67:0/2852033459 learned_addr learned my addr 172.21.15.67:0/2852033459 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f977486f700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770109710 0x7f977010fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976f7fe700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 0x7f9770116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 -- 172.21.15.67:0/2852033459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770109710 msgr2=0x7f977010fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770109710 0x7f977010fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 -- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 msgr2=0x7f9770116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9770070890 0x7f9770116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f976ffff700 1 -- 172.21.15.67:0/2852033459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764005040 con 0x7f977006ff80 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.900+0000 7f977486f700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9770109710 0x7f977010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.901+0000 7f976ffff700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 0x7f9770115d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f976000ebc0 tx=0x7f976000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.901+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760007cc0 con 0x7f977006ff80 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.901+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f976000f040 con 0x7f977006ff80 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.901+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97701109b0 con 0x7f977006ff80 2023-10-07T23:16:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.901+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000c940 con 0x7f977006ff80 2023-10-07T23:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.901+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977011cd20 con 0x7f977006ff80 2023-10-07T23:16:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.902+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f977010b9c0 con 0x7f977006ff80 2023-10-07T23:16:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.905+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f97600090d0 con 0x7f977006ff80 2023-10-07T23:16:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.907+0000 7f976cff9700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f975809b7b0 0x7f975809dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.907+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f97600c4ed0 con 0x7f977006ff80 2023-10-07T23:16:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.907+0000 7f976f7fe700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f975809b7b0 0x7f975809dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.907+0000 7f976cff9700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] conn(0x7f97580a6320 0x7f97580a8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.907+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 --> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97580a8e20 con 0x7f97580a6320 2023-10-07T23:16:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.907+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97600c5260 con 0x7f977006ff80 2023-10-07T23:16:15.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.908+0000 7f977486f700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] conn(0x7f97580a6320 0x7f97580a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:15.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.908+0000 7f976f7fe700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f975809b7b0 0x7f975809dc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9764009fd0 tx=0x7f9764009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:15.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.908+0000 7f977486f700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] conn(0x7f97580a6320 0x7f97580a8750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:15.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.909+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== osd.1 v2:172.21.15.86:6800/3463493653 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f97580a8e20 con 0x7f97580a6320 2023-10-07T23:16:15.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.930+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 --> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9770063f20 con 0x7f97580a6320 2023-10-07T23:16:15.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.931+0000 7f976cff9700 1 -- 172.21.15.67:0/2852033459 <== osd.1 v2:172.21.15.86:6800/3463493653 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9770063f20 con 0x7f97580a6320 2023-10-07T23:16:15.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] conn(0x7f97580a6320 msgr2=0x7f97580a8750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] conn(0x7f97580a6320 0x7f97580a8750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f975809b7b0 msgr2=0x7f975809dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f975809b7b0 0x7f975809dc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9764009fd0 tx=0x7f9764009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 msgr2=0x7f9770115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 --2- 172.21.15.67:0/2852033459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f977006ff80 0x7f9770115d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f976000ebc0 tx=0x7f976000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f977486f700 1 -- 172.21.15.67:0/2852033459 reap_dead start 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 shutdown_connections 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 >> 172.21.15.67:0/2852033459 conn(0x7f9770074320 msgr2=0x7f977010a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 shutdown_connections 2023-10-07T23:16:15.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.932+0000 7f97762d2700 1 -- 172.21.15.67:0/2852033459 wait complete. 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 -- 172.21.15.67:0/2759701729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 msgr2=0x7faacc103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 --2- 172.21.15.67:0/2759701729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 0x7faacc103230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faac80049f0 tx=0x7faac8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 -- 172.21.15.67:0/2759701729 shutdown_connections 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 --2- 172.21.15.67:0/2759701729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc0fd550 0x7faacc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 --2- 172.21.15.67:0/2759701729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 0x7faacc103230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 --2- 172.21.15.67:0/2759701729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc103800 0x7faacc0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 -- 172.21.15.67:0/2759701729 >> 172.21.15.67:0/2759701729 conn(0x7faacc074320 msgr2=0x7faacc074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 -- 172.21.15.67:0/2759701729 shutdown_connections 2023-10-07T23:16:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.826+0000 7faad336f700 1 -- 172.21.15.67:0/2759701729 wait complete. 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 Processor -- start 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 -- start start 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc0fd550 0x7faacc113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 0x7faacc114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc103800 0x7faacc10dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc10e1f0 con 0x7faacc102e60 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc10e360 con 0x7faacc103800 2023-10-07T23:16:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.827+0000 7faad336f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc10e520 con 0x7faacc0fd550 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad190c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc103800 0x7faacc10dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad090a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 0x7faacc114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad190c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc103800 0x7faacc10dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:42724/0 (socket says 172.21.15.67:42724) 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad190c700 1 -- 172.21.15.67:0/3527489233 learned_addr learned my addr 172.21.15.67:0/3527489233 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad110b700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc0fd550 0x7faacc113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad110b700 1 -- 172.21.15.67:0/3527489233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc103800 msgr2=0x7faacc10dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad110b700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc103800 0x7faacc10dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad110b700 1 -- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 msgr2=0x7faacc114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad110b700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc102e60 0x7faacc114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad110b700 1 -- 172.21.15.67:0/3527489233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40048f0 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.828+0000 7faad190c700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faacc103800 0x7faacc10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.829+0000 7faad110b700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc0fd550 0x7faacc113b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faac80049f0 tx=0x7faac8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.829+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800e070 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.829+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac8005040 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.829+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faac8009990 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.830+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac801ce20 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.830+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc11a950 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.831+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faacc0fbea0 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.833+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7faac801c530 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.835+0000 7faac1ffb700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7faab809b770 0x7faab809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.836+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7faac80c69c0 con 0x7faacc0fd550 2023-10-07T23:16:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.836+0000 7faad090a700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7faab809b770 0x7faab809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.836+0000 7faac1ffb700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] conn(0x7faab80a62e0 0x7faab80a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.836+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 --> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faab80a8de0 con 0x7faab80a62e0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.836+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac80c6d50 con 0x7faacc0fd550 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.836+0000 7faad190c700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] conn(0x7faab80a62e0 0x7faab80a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.837+0000 7faad090a700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7faab809b770 0x7faab809dc20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faacc10f3c0 tx=0x7faabc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.837+0000 7faad190c700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] conn(0x7faab80a62e0 0x7faab80a8710 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.838+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== osd.9 v2:172.21.15.67:6826/92905414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7faab80a8de0 con 0x7faab80a62e0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.869+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 --> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faacc063f20 con 0x7faab80a62e0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.870+0000 7faac1ffb700 1 -- 172.21.15.67:0/3527489233 <== osd.9 v2:172.21.15.67:6826/92905414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faacc063f20 con 0x7faab80a62e0 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] conn(0x7faab80a62e0 msgr2=0x7faab80a8710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] conn(0x7faab80a62e0 0x7faab80a8710 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7faab809b770 msgr2=0x7faab809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7faab809b770 0x7faab809dc20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faacc10f3c0 tx=0x7faabc00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc0fd550 msgr2=0x7faacc113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 --2- 172.21.15.67:0/3527489233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faacc0fd550 0x7faacc113b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faac80049f0 tx=0x7faac8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad190c700 1 -- 172.21.15.67:0/3527489233 reap_dead start 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 shutdown_connections 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 >> 172.21.15.67:0/3527489233 conn(0x7faacc074320 msgr2=0x7faacc0fb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.871+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 shutdown_connections 2023-10-07T23:16:16.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:15.872+0000 7faad336f700 1 -- 172.21.15.67:0/3527489233 wait complete. 2023-10-07T23:16:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 -- 172.21.15.67:0/1977028380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973006ffe0 msgr2=0x7f97300703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 --2- 172.21.15.67:0/1977028380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973006ffe0 0x7f97300703b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f972c0049c0 tx=0x7f972c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 -- 172.21.15.67:0/1977028380 shutdown_connections 2023-10-07T23:16:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 --2- 172.21.15.67:0/1977028380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9730109690 0x7f973010ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 --2- 172.21.15.67:0/1977028380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f973006ffe0 0x7f97300703b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 --2- 172.21.15.67:0/1977028380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97300708f0 0x7f9730109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 -- 172.21.15.67:0/1977028380 >> 172.21.15.67:0/1977028380 conn(0x7f97300f89f0 msgr2=0x7f97300fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 -- 172.21.15.67:0/1977028380 shutdown_connections 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.522+0000 7f9738739700 1 -- 172.21.15.67:0/1977028380 wait complete. 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.523+0000 7f9738739700 1 Processor -- start 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.523+0000 7f9738739700 1 -- start start 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9738739700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973006ffe0 0x7f9730115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9738739700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97300708f0 0x7f9730116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9738739700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9730109690 0x7f973010fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9738739700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301103c0 con 0x7f9730109690 2023-10-07T23:16:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9738739700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730110530 con 0x7f973006ffe0 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9738739700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301106f0 con 0x7f97300708f0 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9730109690 0x7f973010fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9730109690 0x7f973010fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57692/0 (socket says 172.21.15.67:57692) 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 -- 172.21.15.67:0/1604097364 learned_addr learned my addr 172.21.15.67:0/1604097364 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9735cd4700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97300708f0 0x7f9730116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f97364d5700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973006ffe0 0x7f9730115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 -- 172.21.15.67:0/1604097364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97300708f0 msgr2=0x7f9730116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97300708f0 0x7f9730116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 -- 172.21.15.67:0/1604097364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973006ffe0 msgr2=0x7f9730115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973006ffe0 0x7f9730115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 -- 172.21.15.67:0/1604097364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9720005040 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f97364d5700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f973006ffe0 0x7f9730115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.524+0000 7f9736cd6700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9730109690 0x7f973010fda0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f972800ec00 tx=0x7f972800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.525+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728007b30 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.525+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f972800f040 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.525+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800caf0 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.525+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c005040 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.525+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973011cb20 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.526+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f973005e340 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.528+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9728007c90 con 0x7f9730109690 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.529+0000 7f9726ffd700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f971c09b7f0 0x7f971c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.529+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f97280c60c0 con 0x7f9730109690 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.529+0000 7f97364d5700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f971c09b7f0 0x7f971c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.529+0000 7f9726ffd700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] conn(0x7f971c0a6360 0x7f971c0a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.529+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 --> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f971c0a8e60 con 0x7f971c0a6360 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.529+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97280c63b0 con 0x7f9730109690 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.530+0000 7f9735cd4700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] conn(0x7f971c0a6360 0x7f971c0a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.530+0000 7f97364d5700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f971c09b7f0 0x7f971c09dca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f972c007f50 tx=0x7f972c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.530+0000 7f9735cd4700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] conn(0x7f971c0a6360 0x7f971c0a8790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.531+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== osd.3 v2:172.21.15.67:6810/2933401597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f971c0a8e60 con 0x7f971c0a6360 2023-10-07T23:16:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.547+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 --> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f973005e770 con 0x7f971c0a6360 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.547+0000 7f9726ffd700 1 -- 172.21.15.67:0/1604097364 <== osd.3 v2:172.21.15.67:6810/2933401597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f973005e770 con 0x7f971c0a6360 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] conn(0x7f971c0a6360 msgr2=0x7f971c0a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] conn(0x7f971c0a6360 0x7f971c0a8790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f971c09b7f0 msgr2=0x7f971c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f971c09b7f0 0x7f971c09dca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f972c007f50 tx=0x7f972c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9730109690 msgr2=0x7f973010fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 --2- 172.21.15.67:0/1604097364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9730109690 0x7f973010fda0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f972800ec00 tx=0x7f972800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9736cd6700 1 -- 172.21.15.67:0/1604097364 reap_dead start 2023-10-07T23:16:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 shutdown_connections 2023-10-07T23:16:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 >> 172.21.15.67:0/1604097364 conn(0x7f97300f89f0 msgr2=0x7f97301010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 shutdown_connections 2023-10-07T23:16:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:16.548+0000 7f9738739700 1 -- 172.21.15.67:0/1604097364 wait complete. 2023-10-07T23:16:17.107 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:16 smithi067 ceph-mon[82723]: pgmap v313: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:17.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:16 smithi123 ceph-mon[122926]: pgmap v313: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:16 smithi086 ceph-mon[122131]: pgmap v313: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:17.547 INFO:teuthology.orchestra.run.smithi067.stdout:34359738446 2023-10-07T23:16:17.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-07T23:16:17.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.291+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3776783928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac0fd550 msgr2=0x7fa6ac109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.291+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3776783928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac109c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4004cd0 tx=0x7fa6a4009ca0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3776783928 shutdown_connections 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3776783928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac102e60 0x7fa6ac103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3776783928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac109c40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3776783928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6ac103800 0x7fa6ac0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3776783928 >> 172.21.15.67:0/3776783928 conn(0x7fa6ac074320 msgr2=0x7fa6ac074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:17.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3776783928 shutdown_connections 2023-10-07T23:16:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.292+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3776783928 wait complete. 2023-10-07T23:16:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.294+0000 7fa6b2aa4700 1 Processor -- start 2023-10-07T23:16:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.294+0000 7fa6b2aa4700 1 -- start start 2023-10-07T23:16:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.295+0000 7fa6b2aa4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.295+0000 7fa6b2aa4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac102e60 0x7fa6ac113be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.295+0000 7fa6b2aa4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6ac103800 0x7fa6ac10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.296+0000 7fa6b0840700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.296+0000 7fa6b0840700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac113660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:42780/0 (socket says 172.21.15.67:42780) 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.296+0000 7fa6b0840700 1 -- 172.21.15.67:0/3768923764 learned_addr learned my addr 172.21.15.67:0/3768923764 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.296+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10e040 con 0x7fa6ac103800 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b1041700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6ac103800 0x7fa6ac10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10e1b0 con 0x7fa6ac0fd550 2023-10-07T23:16:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac10e370 con 0x7fa6ac102e60 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b0840700 1 -- 172.21.15.67:0/3768923764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac102e60 msgr2=0x7fa6ac113be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6abfff700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac102e60 0x7fa6ac113be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b0840700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac102e60 0x7fa6ac113be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b0840700 1 -- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6ac103800 msgr2=0x7fa6ac10d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b0840700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6ac103800 0x7fa6ac10d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.297+0000 7fa6b0840700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a0005040 con 0x7fa6ac0fd550 2023-10-07T23:16:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.298+0000 7fa6abfff700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ac102e60 0x7fa6ac113be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:16:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.298+0000 7fa6b1041700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6ac103800 0x7fa6ac10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.298+0000 7fa6b0840700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac113660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa69c004cb0 tx=0x7fa69c00e500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.298+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c013070 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.298+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69c00e640 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.298+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a40048f0 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.299+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00cb60 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.299+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac070030 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.300+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6ac102480 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.303+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fa69c044aa0 con 0x7fa6ac0fd550 2023-10-07T23:16:17.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.306+0000 7fa6a97fa700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa69409b7f0 0x7fa69409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.306+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fa69c0c95d0 con 0x7fa6ac0fd550 2023-10-07T23:16:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.306+0000 7fa6a97fa700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] conn(0x7fa6940a6360 0x7fa6940a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.307+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6940a8e60 con 0x7fa6940a6360 2023-10-07T23:16:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.307+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa69c0c98c0 con 0x7fa6ac0fd550 2023-10-07T23:16:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.307+0000 7fa6abfff700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa69409b7f0 0x7fa69409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.308+0000 7fa6abfff700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa69409b7f0 0x7fa69409dca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac10ef00 tx=0x7fa6a0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.309+0000 7fa6b1041700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] conn(0x7fa6940a6360 0x7fa6940a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.309+0000 7fa6b1041700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] conn(0x7fa6940a6360 0x7fa6940a8790 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.310+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== osd.10 v2:172.21.15.86:6824/2965800150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa6940a8e60 con 0x7fa6940a6360 2023-10-07T23:16:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.336+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 --> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6ac10e840 con 0x7fa6940a6360 2023-10-07T23:16:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.337+0000 7fa6a97fa700 1 -- 172.21.15.67:0/3768923764 <== osd.10 v2:172.21.15.86:6824/2965800150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa6ac10e840 con 0x7fa6940a6360 2023-10-07T23:16:17.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.337+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] conn(0x7fa6940a6360 msgr2=0x7fa6940a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.337+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] conn(0x7fa6940a6360 0x7fa6940a8790 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa69409b7f0 msgr2=0x7fa69409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa69409b7f0 0x7fa69409dca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac10ef00 tx=0x7fa6a0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac0fd550 msgr2=0x7fa6ac113660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 --2- 172.21.15.67:0/3768923764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ac0fd550 0x7fa6ac113660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa69c004cb0 tx=0x7fa69c00e500 comp rx=0 tx=0).stop 2023-10-07T23:16:17.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b1041700 1 -- 172.21.15.67:0/3768923764 reap_dead start 2023-10-07T23:16:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 shutdown_connections 2023-10-07T23:16:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 >> 172.21.15.67:0/3768923764 conn(0x7fa6ac074320 msgr2=0x7fa6ac100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 shutdown_connections 2023-10-07T23:16:17.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.338+0000 7fa6b2aa4700 1 -- 172.21.15.67:0/3768923764 wait complete. 2023-10-07T23:16:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 -- 172.21.15.67:0/1671442142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065fc0 msgr2=0x7f41bc109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 --2- 172.21.15.67:0/1671442142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065fc0 0x7f41bc109c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009d60 tx=0x7f41b4004d10 comp rx=0 tx=0).stop 2023-10-07T23:16:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 -- 172.21.15.67:0/1671442142 shutdown_connections 2023-10-07T23:16:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 --2- 172.21.15.67:0/1671442142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065fc0 0x7f41bc109c50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 --2- 172.21.15.67:0/1671442142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc102ee0 0x7f41bc1032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 --2- 172.21.15.67:0/1671442142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41bc065630 0x7f41bc065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 -- 172.21.15.67:0/1671442142 >> 172.21.15.67:0/1671442142 conn(0x7f41bc071c80 msgr2=0x7f41bc072080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 -- 172.21.15.67:0/1671442142 shutdown_connections 2023-10-07T23:16:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.612+0000 7f41c2860700 1 -- 172.21.15.67:0/1671442142 wait complete. 2023-10-07T23:16:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.613+0000 7f41c2860700 1 Processor -- start 2023-10-07T23:16:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 -- start start 2023-10-07T23:16:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065630 0x7f41bc115a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc065fc0 0x7f41bc115f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41bc102ee0 0x7f41bc10faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc110140 con 0x7f41bc065fc0 2023-10-07T23:16:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1102b0 con 0x7f41bc065630 2023-10-07T23:16:17.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c2860700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc110470 con 0x7f41bc102ee0 2023-10-07T23:16:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc065fc0 0x7f41bc115f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41bb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc065fc0 0x7f41bc115f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57752/0 (socket says 172.21.15.67:57752) 2023-10-07T23:16:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41bb7fe700 1 -- 172.21.15.67:0/4181213422 learned_addr learned my addr 172.21.15.67:0/4181213422 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41bbfff700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065630 0x7f41bc115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.614+0000 7f41c0dfd700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41bc102ee0 0x7f41bc10faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41bb7fe700 1 -- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41bc102ee0 msgr2=0x7f41bc10faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41bb7fe700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41bc102ee0 0x7f41bc10faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41bb7fe700 1 -- 172.21.15.67:0/4181213422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065630 msgr2=0x7f41bc115a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41bb7fe700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc065630 0x7f41bc115a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41bb7fe700 1 -- 172.21.15.67:0/4181213422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4009940 con 0x7f41bc065fc0 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41bb7fe700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc065fc0 0x7f41bc115f40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41b000ebc0 tx=0x7f41b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0007cc0 con 0x7f41bc065fc0 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc1106f0 con 0x7f41bc065fc0 2023-10-07T23:16:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41b000f040 con 0x7f41bc065fc0 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b000c840 con 0x7f41bc065fc0 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.615+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc11ca30 con 0x7f41bc065fc0 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.616+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41bc0fc990 con 0x7f41bc065fc0 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.618+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f41b000cc70 con 0x7f41bc065fc0 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.619+0000 7f41b8ff9700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41a409b7f0 0x7f41a409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f41b00c4c50 con 0x7f41bc065fc0 2023-10-07T23:16:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41bbfff700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41a409b7f0 0x7f41a409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41b8ff9700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] conn(0x7f41a40a6360 0x7f41a40a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 --> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41a40a8e60 con 0x7f41a40a6360 2023-10-07T23:16:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b00c9220 con 0x7f41bc065fc0 2023-10-07T23:16:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41bbfff700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41a409b7f0 0x7f41a409dca0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009b80 tx=0x7f41ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.620+0000 7f41c0dfd700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] conn(0x7f41a40a6360 0x7f41a40a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.621+0000 7f41c0dfd700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] conn(0x7f41a40a6360 0x7f41a40a8790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.622+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== osd.11 v2:172.21.15.123:6824/2458295261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f41a40a8e60 con 0x7f41a40a6360 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.638+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 --> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41bc063f20 con 0x7f41a40a6360 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41b8ff9700 1 -- 172.21.15.67:0/4181213422 <== osd.11 v2:172.21.15.123:6824/2458295261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f41bc063f20 con 0x7f41a40a6360 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] conn(0x7f41a40a6360 msgr2=0x7f41a40a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] conn(0x7f41a40a6360 0x7f41a40a8790 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41a409b7f0 msgr2=0x7f41a409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41a409b7f0 0x7f41a409dca0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009b80 tx=0x7f41ac00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc065fc0 msgr2=0x7f41bc115f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 --2- 172.21.15.67:0/4181213422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41bc065fc0 0x7f41bc115f40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41b000ebc0 tx=0x7f41b000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c0dfd700 1 -- 172.21.15.67:0/4181213422 reap_dead start 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.639+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 shutdown_connections 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.640+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 >> 172.21.15.67:0/4181213422 conn(0x7f41bc071c80 msgr2=0x7f41bc0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.640+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 shutdown_connections 2023-10-07T23:16:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.640+0000 7f41c2860700 1 -- 172.21.15.67:0/4181213422 wait complete. 2023-10-07T23:16:18.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 -- 172.21.15.67:0/255056540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00102e60 msgr2=0x7fca00103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:18.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 --2- 172.21.15.67:0/255056540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00102e60 0x7fca00103230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00049f0 tx=0x7fc9f0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 -- 172.21.15.67:0/255056540 shutdown_connections 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 --2- 172.21.15.67:0/255056540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 0x7fca00109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 --2- 172.21.15.67:0/255056540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00102e60 0x7fca00103230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 --2- 172.21.15.67:0/255056540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00103800 0x7fca000fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 -- 172.21.15.67:0/255056540 >> 172.21.15.67:0/255056540 conn(0x7fca00074320 msgr2=0x7fca00074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 -- 172.21.15.67:0/255056540 shutdown_connections 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.233+0000 7fca050a5700 1 -- 172.21.15.67:0/255056540 wait complete. 2023-10-07T23:16:18.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.235+0000 7fca050a5700 1 Processor -- start 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.235+0000 7fca050a5700 1 -- start start 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.236+0000 7fca050a5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 0x7fca00113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.236+0000 7fca050a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00102e60 0x7fca00113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fca050a5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00103800 0x7fca0010db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fc9fed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 0x7fca00113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fc9fed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 0x7fca00113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:42754/0 (socket says 172.21.15.67:42754) 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fc9fe59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00102e60 0x7fca00113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:18.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fca050a5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0010e170 con 0x7fca00102e60 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fca050a5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0010e2e0 con 0x7fca000fd550 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fca050a5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0010e4a0 con 0x7fca00103800 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fc9fed9d700 1 -- 172.21.15.67:0/2709654490 learned_addr learned my addr 172.21.15.67:0/2709654490 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fc9fe59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00102e60 0x7fca00113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57714/0 (socket says 172.21.15.67:57714) 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.237+0000 7fc9ff59e700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00103800 0x7fca0010db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fed9d700 1 -- 172.21.15.67:0/2709654490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00103800 msgr2=0x7fca0010db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fed9d700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca00103800 0x7fca0010db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:18.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fed9d700 1 -- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00102e60 msgr2=0x7fca00113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fed9d700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00102e60 0x7fca00113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fed9d700 1 -- 172.21.15.67:0/2709654490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f0005040 con 0x7fca000fd550 2023-10-07T23:16:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fe59c700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca00102e60 0x7fca00113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.238+0000 7fc9fed9d700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 0x7fca00113aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009fd0 tx=0x7fc9f0009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.239+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000e070 con 0x7fca000fd550 2023-10-07T23:16:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.239+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9f00095a0 con 0x7fca000fd550 2023-10-07T23:16:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.239+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0013be0 con 0x7fca000fd550 2023-10-07T23:16:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.239+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0010e720 con 0x7fca000fd550 2023-10-07T23:16:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.239+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca00070030 con 0x7fca000fd550 2023-10-07T23:16:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.240+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca00104670 con 0x7fca000fd550 2023-10-07T23:16:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.244+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fc9f0013d40 con 0x7fca000fd550 2023-10-07T23:16:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.246+0000 7fc9e77fe700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc9e809fc80 0x7fc9e80a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.247+0000 7fc9fe59c700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc9e809fc80 0x7fc9e80a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.248+0000 7fc9fe59c700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc9e809fc80 0x7fc9e80a2130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fca0010f340 tx=0x7fc9f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.248+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fc9f00c6650 con 0x7fca000fd550 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.248+0000 7fc9e77fe700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] conn(0x7fc9e80aa7f0 0x7fc9e80acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.249+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 --> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9e80ad2f0 con 0x7fc9e80aa7f0 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.249+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9f00c6980 con 0x7fca000fd550 2023-10-07T23:16:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.250+0000 7fc9ff59e700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] conn(0x7fc9e80aa7f0 0x7fc9e80acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.251+0000 7fc9ff59e700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] conn(0x7fc9e80aa7f0 0x7fc9e80acc20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.252+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== osd.6 v2:172.21.15.67:6818/3799339954 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc9e80ad2f0 con 0x7fc9e80aa7f0 2023-10-07T23:16:18.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.281+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 --> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca00063f20 con 0x7fc9e80aa7f0 2023-10-07T23:16:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.282+0000 7fc9e77fe700 1 -- 172.21.15.67:0/2709654490 <== osd.6 v2:172.21.15.67:6818/3799339954 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fca00063f20 con 0x7fc9e80aa7f0 2023-10-07T23:16:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.282+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] conn(0x7fc9e80aa7f0 msgr2=0x7fc9e80acc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.282+0000 7fca050a5700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] conn(0x7fc9e80aa7f0 0x7fc9e80acc20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.283+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc9e809fc80 msgr2=0x7fc9e80a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.283+0000 7fca050a5700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc9e809fc80 0x7fc9e80a2130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fca0010f340 tx=0x7fc9f400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 msgr2=0x7fca00113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fca050a5700 1 --2- 172.21.15.67:0/2709654490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca000fd550 0x7fca00113aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009fd0 tx=0x7fc9f0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fc9ff59e700 1 -- 172.21.15.67:0/2709654490 reap_dead start 2023-10-07T23:16:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 shutdown_connections 2023-10-07T23:16:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 >> 172.21.15.67:0/2709654490 conn(0x7fca00074320 msgr2=0x7fca00103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 shutdown_connections 2023-10-07T23:16:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:17.284+0000 7fca050a5700 1 -- 172.21.15.67:0/2709654490 wait complete. 2023-10-07T23:16:18.391 INFO:teuthology.orchestra.run.smithi067.stdout:55834574921 2023-10-07T23:16:18.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-07T23:16:18.756 INFO:teuthology.orchestra.run.smithi067.stdout:343597383701 2023-10-07T23:16:18.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-07T23:16:19.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:18 smithi067 ceph-mon[82723]: pgmap v314: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-07T23:16:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:18 smithi123 ceph-mon[122926]: pgmap v314: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-07T23:16:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:18 smithi086 ceph-mon[122131]: pgmap v314: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-10-07T23:16:19.516 INFO:teuthology.orchestra.run.smithi067.stdout:287762808864 2023-10-07T23:16:19.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-07T23:16:19.843 INFO:teuthology.orchestra.run.smithi067.stdout:77309411395 2023-10-07T23:16:19.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.957+0000 7fc2339bc700 1 -- 172.21.15.67:0/1431004328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c103800 msgr2=0x7fc22c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.957+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1431004328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c103800 0x7fc22c0fce70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0049f0 tx=0x7fc21c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 -- 172.21.15.67:0/1431004328 shutdown_connections 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1431004328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc22c0fd550 0x7fc22c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1431004328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c102e60 0x7fc22c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1431004328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c103800 0x7fc22c0fce70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 -- 172.21.15.67:0/1431004328 >> 172.21.15.67:0/1431004328 conn(0x7fc22c074320 msgr2=0x7fc22c074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 -- 172.21.15.67:0/1431004328 shutdown_connections 2023-10-07T23:16:19.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.958+0000 7fc2339bc700 1 -- 172.21.15.67:0/1431004328 wait complete. 2023-10-07T23:16:19.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.959+0000 7fc2339bc700 1 Processor -- start 2023-10-07T23:16:19.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.959+0000 7fc2339bc700 1 -- start start 2023-10-07T23:16:19.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc2339bc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c0fd550 0x7fc22c1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:19.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc2339bc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc22c102e60 0x7fc22c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:19.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc2339bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c103800 0x7fc22c10db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:19.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc2339bc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c10e190 con 0x7fc22c0fd550 2023-10-07T23:16:19.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc2339bc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c10e300 con 0x7fc22c102e60 2023-10-07T23:16:19.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc2339bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c10e4c0 con 0x7fc22c103800 2023-10-07T23:16:19.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc231758700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c0fd550 0x7fc22c1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:19.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc231758700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c0fd550 0x7fc22c1138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57772/0 (socket says 172.21.15.67:57772) 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc231758700 1 -- 172.21.15.67:0/1594404599 learned_addr learned my addr 172.21.15.67:0/1594404599 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc230f57700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc22c102e60 0x7fc22c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.960+0000 7fc231f59700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c103800 0x7fc22c10db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc230f57700 1 -- 172.21.15.67:0/1594404599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c103800 msgr2=0x7fc22c10db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc230f57700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc22c103800 0x7fc22c10db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc230f57700 1 -- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c0fd550 msgr2=0x7fc22c1138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc230f57700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc22c0fd550 0x7fc22c1138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:19.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc230f57700 1 -- 172.21.15.67:0/1594404599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228005040 con 0x7fc22c102e60 2023-10-07T23:16:19.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc230f57700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc22c102e60 0x7fc22c113df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc21c013460 tx=0x7fc21c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:19.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.961+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c020070 con 0x7fc22c102e60 2023-10-07T23:16:19.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.962+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c005040 con 0x7fc22c102e60 2023-10-07T23:16:19.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.962+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc21c0098d0 con 0x7fc22c102e60 2023-10-07T23:16:19.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.962+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00e070 con 0x7fc22c102e60 2023-10-07T23:16:19.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.962+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c06fe70 con 0x7fc22c102e60 2023-10-07T23:16:19.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.963+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc22c1046b0 con 0x7fc22c102e60 2023-10-07T23:16:19.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.967+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fc21c012920 con 0x7fc22c102e60 2023-10-07T23:16:19.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.969+0000 7fc221ffb700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc21809b7f0 0x7fc21809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:19.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fc21c0c6ab0 con 0x7fc22c102e60 2023-10-07T23:16:19.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc231758700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc21809b7f0 0x7fc21809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:19.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc221ffb700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] conn(0x7fc2180a6360 0x7fc2180a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:19.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 --> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2180a8e60 con 0x7fc2180a6360 2023-10-07T23:16:19.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21c0c6de0 con 0x7fc22c102e60 2023-10-07T23:16:19.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc231758700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc21809b7f0 0x7fc21809dca0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2280044e0 tx=0x7fc22800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:19.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.970+0000 7fc231f59700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] conn(0x7fc2180a6360 0x7fc2180a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:19.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.971+0000 7fc231f59700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] conn(0x7fc2180a6360 0x7fc2180a8790 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:19.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.972+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== osd.7 v2:172.21.15.86:6816/3974856027 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc2180a8e60 con 0x7fc2180a6360 2023-10-07T23:16:19.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.990+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 --> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc22c063f20 con 0x7fc2180a6360 2023-10-07T23:16:19.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.991+0000 7fc221ffb700 1 -- 172.21.15.67:0/1594404599 <== osd.7 v2:172.21.15.86:6816/3974856027 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc22c063f20 con 0x7fc2180a6360 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.991+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] conn(0x7fc2180a6360 msgr2=0x7fc2180a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] conn(0x7fc2180a6360 0x7fc2180a8790 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc21809b7f0 msgr2=0x7fc21809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc21809b7f0 0x7fc21809dca0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2280044e0 tx=0x7fc22800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc22c102e60 msgr2=0x7fc22c113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 --2- 172.21.15.67:0/1594404599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc22c102e60 0x7fc22c113df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc21c013460 tx=0x7fc21c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc231f59700 1 -- 172.21.15.67:0/1594404599 reap_dead start 2023-10-07T23:16:19.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 shutdown_connections 2023-10-07T23:16:19.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.992+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 >> 172.21.15.67:0/1594404599 conn(0x7fc22c074320 msgr2=0x7fc22c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:19.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.993+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 shutdown_connections 2023-10-07T23:16:19.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:19.993+0000 7fc2339bc700 1 -- 172.21.15.67:0/1594404599 wait complete. 2023-10-07T23:16:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 -- 172.21.15.67:0/4041057822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa1006e8f0 msgr2=0x7ffa1006ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 --2- 172.21.15.67:0/4041057822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa1006e8f0 0x7ffa1006ecc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa00004900 tx=0x7ffa00009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 -- 172.21.15.67:0/4041057822 shutdown_connections 2023-10-07T23:16:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 --2- 172.21.15.67:0/4041057822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa1006e8f0 0x7ffa1006ecc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 --2- 172.21.15.67:0/4041057822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa10108750 0x7ffa10118ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 --2- 172.21.15.67:0/4041057822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa1006f290 0x7ffa10108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 -- 172.21.15.67:0/4041057822 >> 172.21.15.67:0/4041057822 conn(0x7ffa1006a3d0 msgr2=0x7ffa1006a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:20.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 -- 172.21.15.67:0/4041057822 shutdown_connections 2023-10-07T23:16:20.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.159+0000 7ffa1585e700 1 -- 172.21.15.67:0/4041057822 wait complete. 2023-10-07T23:16:20.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.160+0000 7ffa1585e700 1 Processor -- start 2023-10-07T23:16:20.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.160+0000 7ffa1585e700 1 -- start start 2023-10-07T23:16:20.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa1585e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1006e8f0 0x7ffa10122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa1585e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa1006f290 0x7ffa10122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa1585e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa10108750 0x7ffa1011c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa0e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa1006f290 0x7ffa10122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa1585e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1011cfd0 con 0x7ffa1006f290 2023-10-07T23:16:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa0e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa1006f290 0x7ffa10122dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57792/0 (socket says 172.21.15.67:57792) 2023-10-07T23:16:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa0e7fc700 1 -- 172.21.15.67:0/1194487660 learned_addr learned my addr 172.21.15.67:0/1194487660 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa0effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1006e8f0 0x7ffa10122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1011d140 con 0x7ffa10108750 2023-10-07T23:16:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.161+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1011d300 con 0x7ffa1006e8f0 2023-10-07T23:16:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0f7fe700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa10108750 0x7ffa1011c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0effd700 1 -- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa10108750 msgr2=0x7ffa1011c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0effd700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa10108750 0x7ffa1011c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0effd700 1 -- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa1006f290 msgr2=0x7ffa10122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0effd700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa1006f290 0x7ffa10122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0effd700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00005040 con 0x7ffa1006e8f0 2023-10-07T23:16:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa0effd700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1006e8f0 0x7ffa10122890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009f80 tx=0x7ffa00009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000e040 con 0x7ffa1006e8f0 2023-10-07T23:16:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1011d580 con 0x7ffa1006e8f0 2023-10-07T23:16:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.162+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa00000c90 con 0x7ffa1006e8f0 2023-10-07T23:16:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.163+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000123c0 con 0x7ffa1006e8f0 2023-10-07T23:16:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.163+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa10129850 con 0x7ffa1006e8f0 2023-10-07T23:16:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.164+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa1010bc30 con 0x7ffa1006e8f0 2023-10-07T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.166+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7ffa0001c430 con 0x7ffa1006e8f0 2023-10-07T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.167+0000 7ff9f77fe700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff9f809b7f0 0x7ff9f809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.167+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7ffa000c68d0 con 0x7ffa1006e8f0 2023-10-07T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.167+0000 7ffa0e7fc700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff9f809b7f0 0x7ff9f809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.168+0000 7ff9f77fe700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] conn(0x7ff9f80a6360 0x7ff9f80a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.168+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9f80a8e60 con 0x7ff9f80a6360 2023-10-07T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.168+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa000c6bc0 con 0x7ffa1006e8f0 2023-10-07T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.168+0000 7ffa0f7fe700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] conn(0x7ff9f80a6360 0x7ff9f80a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.168+0000 7ffa0e7fc700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff9f809b7f0 0x7ff9f809dca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa1011e0c0 tx=0x7ffa0400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.169+0000 7ffa0f7fe700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] conn(0x7ff9f80a6360 0x7ff9f80a8790 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.169+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== osd.4 v2:172.21.15.86:6808/4225579249 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff9f80a8e60 con 0x7ff9f80a6360 2023-10-07T23:16:20.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.192+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 --> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa10063f20 con 0x7ff9f80a6360 2023-10-07T23:16:20.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.192+0000 7ff9f77fe700 1 -- 172.21.15.67:0/1194487660 <== osd.4 v2:172.21.15.86:6808/4225579249 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffa10063f20 con 0x7ff9f80a6360 2023-10-07T23:16:20.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] conn(0x7ff9f80a6360 msgr2=0x7ff9f80a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] conn(0x7ff9f80a6360 0x7ff9f80a8790 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff9f809b7f0 msgr2=0x7ff9f809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff9f809b7f0 0x7ff9f809dca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa1011e0c0 tx=0x7ffa0400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1006e8f0 msgr2=0x7ffa10122890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 --2- 172.21.15.67:0/1194487660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa1006e8f0 0x7ffa10122890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009f80 tx=0x7ffa00009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa0f7fe700 1 -- 172.21.15.67:0/1194487660 reap_dead start 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 shutdown_connections 2023-10-07T23:16:20.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.193+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 >> 172.21.15.67:0/1194487660 conn(0x7ffa1006a3d0 msgr2=0x7ffa1010b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:20.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.194+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 shutdown_connections 2023-10-07T23:16:20.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.194+0000 7ffa1585e700 1 -- 172.21.15.67:0/1194487660 wait complete. 2023-10-07T23:16:20.224 INFO:teuthology.orchestra.run.smithi067.stdout:197568495664 2023-10-07T23:16:20.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-07T23:16:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 -- 172.21.15.67:0/3044821499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 msgr2=0x7fc70c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 --2- 172.21.15.67:0/3044821499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 0x7fc70c109c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc704009d90 tx=0x7fc704004d10 comp rx=0 tx=0).stop 2023-10-07T23:16:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 -- 172.21.15.67:0/3044821499 shutdown_connections 2023-10-07T23:16:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 --2- 172.21.15.67:0/3044821499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c102e60 0x7fc70c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 --2- 172.21.15.67:0/3044821499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 0x7fc70c109c40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 --2- 172.21.15.67:0/3044821499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc70c103800 0x7fc70c0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 -- 172.21.15.67:0/3044821499 >> 172.21.15.67:0/3044821499 conn(0x7fc70c074320 msgr2=0x7fc70c074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 -- 172.21.15.67:0/3044821499 shutdown_connections 2023-10-07T23:16:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.313+0000 7fc7120c4700 1 -- 172.21.15.67:0/3044821499 wait complete. 2023-10-07T23:16:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.314+0000 7fc7120c4700 1 Processor -- start 2023-10-07T23:16:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.315+0000 7fc7120c4700 1 -- start start 2023-10-07T23:16:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.316+0000 7fc7120c4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 0x7fc70c113880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.317+0000 7fc7120c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc70c102e60 0x7fc70c113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.317+0000 7fc7120c4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c103800 0x7fc70c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.317+0000 7fc70b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 0x7fc70c113880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.317+0000 7fc70b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 0x7fc70c113880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57800/0 (socket says 172.21.15.67:57800) 2023-10-07T23:16:20.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.317+0000 7fc70b7fe700 1 -- 172.21.15.67:0/1002016940 learned_addr learned my addr 172.21.15.67:0/1002016940 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:20.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.317+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c10dfc0 con 0x7fc70c0fd550 2023-10-07T23:16:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70affd700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc70c102e60 0x7fc70c113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70bfff700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c103800 0x7fc70c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c10e130 con 0x7fc70c103800 2023-10-07T23:16:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c10e2f0 con 0x7fc70c102e60 2023-10-07T23:16:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70bfff700 1 -- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc70c102e60 msgr2=0x7fc70c113dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70bfff700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc70c102e60 0x7fc70c113dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70bfff700 1 -- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 msgr2=0x7fc70c113880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70bfff700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc70c0fd550 0x7fc70c113880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.318+0000 7fc70bfff700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc704009940 con 0x7fc70c103800 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.319+0000 7fc70bfff700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c103800 0x7fc70c10d950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc704009d90 tx=0x7fc704004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.319+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400e070 con 0x7fc70c103800 2023-10-07T23:16:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.319+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc704008ec0 con 0x7fc70c103800 2023-10-07T23:16:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.320+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c10e570 con 0x7fc70c103800 2023-10-07T23:16:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.320+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040256b0 con 0x7fc70c103800 2023-10-07T23:16:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.320+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c10e7d0 con 0x7fc70c103800 2023-10-07T23:16:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.321+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc70c102480 con 0x7fc70c103800 2023-10-07T23:16:20.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.325+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fc70401b620 con 0x7fc70c103800 2023-10-07T23:16:20.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.327+0000 7fc6f3fff700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc6f409b7f0 0x7fc6f409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.328+0000 7fc70b7fe700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc6f409b7f0 0x7fc6f409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.328+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fc7040ca9a0 con 0x7fc70c103800 2023-10-07T23:16:20.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.328+0000 7fc6f3fff700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] conn(0x7fc6f40a6360 0x7fc6f40a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:20.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.328+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6f40a8e60 con 0x7fc6f40a6360 2023-10-07T23:16:20.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.328+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7040cf220 con 0x7fc70c103800 2023-10-07T23:16:20.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.329+0000 7fc70b7fe700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc6f409b7f0 0x7fc6f409dca0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc008fd0 tx=0x7fc6fc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:20.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.329+0000 7fc70affd700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] conn(0x7fc6f40a6360 0x7fc6f40a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:20.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.330+0000 7fc70affd700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] conn(0x7fc6f40a6360 0x7fc6f40a8790 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:20.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.331+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== osd.8 v2:172.21.15.123:6816/1197944476 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc6f40a8e60 con 0x7fc6f40a6360 2023-10-07T23:16:20.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.356+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 --> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc70c063f20 con 0x7fc6f40a6360 2023-10-07T23:16:20.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.357+0000 7fc6f3fff700 1 -- 172.21.15.67:0/1002016940 <== osd.8 v2:172.21.15.123:6816/1197944476 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc70c063f20 con 0x7fc6f40a6360 2023-10-07T23:16:20.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.359+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] conn(0x7fc6f40a6360 msgr2=0x7fc6f40a8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.359+0000 7fc7120c4700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] conn(0x7fc6f40a6360 0x7fc6f40a8790 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:20.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.359+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc6f409b7f0 msgr2=0x7fc6f409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc6f409b7f0 0x7fc6f409dca0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc008fd0 tx=0x7fc6fc00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:20.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c103800 msgr2=0x7fc70c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:20.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 --2- 172.21.15.67:0/1002016940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c103800 0x7fc70c10d950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc704009d90 tx=0x7fc704004d10 comp rx=0 tx=0).stop 2023-10-07T23:16:20.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc70bfff700 1 -- 172.21.15.67:0/1002016940 reap_dead start 2023-10-07T23:16:20.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 shutdown_connections 2023-10-07T23:16:20.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 >> 172.21.15.67:0/1002016940 conn(0x7fc70c074320 msgr2=0x7fc70c100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:20.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 shutdown_connections 2023-10-07T23:16:20.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.360+0000 7fc7120c4700 1 -- 172.21.15.67:0/1002016940 wait complete. 2023-10-07T23:16:20.595 INFO:teuthology.orchestra.run.smithi067.stdout:309237645338 2023-10-07T23:16:20.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-07T23:16:20.939 INFO:teuthology.orchestra.run.smithi067.stdout:107374182462 2023-10-07T23:16:20.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-07T23:16:21.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 -- 172.21.15.67:0/4220351734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae28102e60 msgr2=0x7fae28103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:21.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/4220351734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae28102e60 0x7fae28103230 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae240049f0 tx=0x7fae24009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:21.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 -- 172.21.15.67:0/4220351734 shutdown_connections 2023-10-07T23:16:21.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/4220351734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae280fd550 0x7fae28109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/4220351734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae28102e60 0x7fae28103230 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/4220351734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae28103800 0x7fae280fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 -- 172.21.15.67:0/4220351734 >> 172.21.15.67:0/4220351734 conn(0x7fae28074320 msgr2=0x7fae28074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 -- 172.21.15.67:0/4220351734 shutdown_connections 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.783+0000 7fae2f3c2700 1 -- 172.21.15.67:0/4220351734 wait complete. 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.784+0000 7fae2f3c2700 1 Processor -- start 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.784+0000 7fae2f3c2700 1 -- start start 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2f3c2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae280fd550 0x7fae2806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2f3c2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae28102e60 0x7fae28070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2f3c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae28103800 0x7fae280709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2f3c2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2811c150 con 0x7fae280fd550 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2f3c2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2811c2c0 con 0x7fae28102e60 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2f3c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2811c480 con 0x7fae28103800 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae280fd550 0x7fae2806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae280fd550 0x7fae2806ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57826/0 (socket says 172.21.15.67:57826) 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 -- 172.21.15.67:0/1504515430 learned_addr learned my addr 172.21.15.67:0/1504515430 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2c95d700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae28102e60 0x7fae28070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d95f700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae28103800 0x7fae280709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 -- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae28103800 msgr2=0x7fae280709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae28103800 0x7fae280709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 -- 172.21.15.67:0/1504515430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae28102e60 msgr2=0x7fae28070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae28102e60 0x7fae28070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d15e700 1 -- 172.21.15.67:0/1504515430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24005040 con 0x7fae280fd550 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.785+0000 7fae2d95f700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae28103800 0x7fae280709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.786+0000 7fae2d15e700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae280fd550 0x7fae2806ff40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae24009400 tx=0x7fae24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.786+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400e070 con 0x7fae280fd550 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.786+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2811c700 con 0x7fae280fd550 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.786+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae24009710 con 0x7fae280fd550 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.786+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24012630 con 0x7fae280fd550 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.786+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2811cbe0 con 0x7fae280fd550 2023-10-07T23:16:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.787+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae280fbe70 con 0x7fae280fd550 2023-10-07T23:16:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.791+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fae24012a60 con 0x7fae280fd550 2023-10-07T23:16:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.792+0000 7fae1dffb700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fae1409b770 0x7fae1409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.792+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fae240c7370 con 0x7fae280fd550 2023-10-07T23:16:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.792+0000 7fae2c95d700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fae1409b770 0x7fae1409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.793+0000 7fae1dffb700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] conn(0x7fae140a62e0 0x7fae140a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.793+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 --> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae140a8de0 con 0x7fae140a62e0 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.793+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae240c7700 con 0x7fae280fd550 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.793+0000 7fae2d95f700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] conn(0x7fae140a62e0 0x7fae140a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.793+0000 7fae2c95d700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fae1409b770 0x7fae1409dc20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae28071730 tx=0x7fae1800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.794+0000 7fae2d95f700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] conn(0x7fae140a62e0 0x7fae140a8710 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.795+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== osd.5 v2:172.21.15.123:6808/476642923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fae140a8de0 con 0x7fae140a62e0 2023-10-07T23:16:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.814+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 --> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae28063f20 con 0x7fae140a62e0 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.815+0000 7fae1dffb700 1 -- 172.21.15.67:0/1504515430 <== osd.5 v2:172.21.15.123:6808/476642923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fae28063f20 con 0x7fae140a62e0 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.815+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] conn(0x7fae140a62e0 msgr2=0x7fae140a8710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.815+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] conn(0x7fae140a62e0 0x7fae140a8710 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fae1409b770 msgr2=0x7fae1409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fae1409b770 0x7fae1409dc20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae28071730 tx=0x7fae1800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae280fd550 msgr2=0x7fae2806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 --2- 172.21.15.67:0/1504515430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae280fd550 0x7fae2806ff40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae24009400 tx=0x7fae24009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2d95f700 1 -- 172.21.15.67:0/1504515430 reap_dead start 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 shutdown_connections 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 >> 172.21.15.67:0/1504515430 conn(0x7fae28074320 msgr2=0x7fae280fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 shutdown_connections 2023-10-07T23:16:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:20.816+0000 7fae2f3c2700 1 -- 172.21.15.67:0/1504515430 wait complete. 2023-10-07T23:16:21.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:20 smithi067 ceph-mon[82723]: pgmap v315: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2023-10-07T23:16:21.237 INFO:teuthology.orchestra.run.smithi067.stdout:253403070503 2023-10-07T23:16:21.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-07T23:16:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:20 smithi123 ceph-mon[122926]: pgmap v315: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2023-10-07T23:16:21.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:20 smithi086 ceph-mon[122131]: pgmap v315: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2023-10-07T23:16:21.600 INFO:teuthology.orchestra.run.smithi067.stdout:141733920826 2023-10-07T23:16:21.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-07T23:16:21.825 INFO:teuthology.orchestra.run.smithi067.stdout:163208757302 2023-10-07T23:16:21.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-07T23:16:21.888 INFO:teuthology.orchestra.run.smithi067.stdout:219043332138 2023-10-07T23:16:21.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-07T23:16:22.074 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:22.443 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:22.567 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:23.117 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:22 smithi067 ceph-mon[82723]: pgmap v316: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-10-07T23:16:23.221 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:23.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:22 smithi123 ceph-mon[122926]: pgmap v316: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-10-07T23:16:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:22 smithi086 ceph-mon[122131]: pgmap v316: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-10-07T23:16:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:24.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:24 smithi067 ceph-mon[82723]: pgmap v317: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2023-10-07T23:16:25.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:24 smithi123 ceph-mon[122926]: pgmap v317: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2023-10-07T23:16:25.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:24 smithi086 ceph-mon[122131]: pgmap v317: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2023-10-07T23:16:25.813 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:25.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.871+0000 7f0ddecde700 1 -- 172.21.15.67:0/1819893169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 msgr2=0x7f0dd80fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:25.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.871+0000 7f0ddecde700 1 --2- 172.21.15.67:0/1819893169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 0x7f0dd80fce70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4004930 tx=0x7f0dc4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:25.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.872+0000 7f0ddecde700 1 -- 172.21.15.67:0/1819893169 shutdown_connections 2023-10-07T23:16:25.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.872+0000 7f0ddecde700 1 --2- 172.21.15.67:0/1819893169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd8102e60 0x7f0dd8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:25.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.872+0000 7f0ddecde700 1 --2- 172.21.15.67:0/1819893169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd80fd550 0x7f0dd8109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:25.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.872+0000 7f0ddecde700 1 --2- 172.21.15.67:0/1819893169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 0x7f0dd80fce70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:25.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.872+0000 7f0ddecde700 1 -- 172.21.15.67:0/1819893169 >> 172.21.15.67:0/1819893169 conn(0x7f0dd8074320 msgr2=0x7f0dd8074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:25.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.874+0000 7f0ddecde700 1 -- 172.21.15.67:0/1819893169 shutdown_connections 2023-10-07T23:16:25.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.874+0000 7f0ddecde700 1 -- 172.21.15.67:0/1819893169 wait complete. 2023-10-07T23:16:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.875+0000 7f0ddecde700 1 Processor -- start 2023-10-07T23:16:25.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.876+0000 7f0ddecde700 1 -- start start 2023-10-07T23:16:25.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.876+0000 7f0ddecde700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 0x7f0dd81128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:25.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.876+0000 7f0ddecde700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd8102e60 0x7f0dd810b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:25.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddecde700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 0x7f0dd810bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddca7a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 0x7f0dd81128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0dd7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd8102e60 0x7f0dd810b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddca7a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 0x7f0dd81128c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:33918/0 (socket says 172.21.15.67:33918) 2023-10-07T23:16:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddca7a700 1 -- 172.21.15.67:0/1694693431 learned_addr learned my addr 172.21.15.67:0/1694693431 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddecde700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd810c700 con 0x7f0dd8103800 2023-10-07T23:16:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddecde700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8110470 con 0x7f0dd80fd550 2023-10-07T23:16:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.877+0000 7f0ddecde700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8110630 con 0x7f0dd8102e60 2023-10-07T23:16:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.878+0000 7f0ddca7a700 1 -- 172.21.15.67:0/1694693431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd8102e60 msgr2=0x7f0dd810b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.878+0000 7f0ddca7a700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd8102e60 0x7f0dd810b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.878+0000 7f0ddca7a700 1 -- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 msgr2=0x7f0dd810bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:25.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.878+0000 7f0ddca7a700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 0x7f0dd810bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.878+0000 7f0ddca7a700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4005040 con 0x7f0dd80fd550 2023-10-07T23:16:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.878+0000 7f0ddca7a700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 0x7f0dd81128c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0041d0 tx=0x7f0dcc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.879+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc007cc0 con 0x7f0dd80fd550 2023-10-07T23:16:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.879+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dcc00f040 con 0x7f0dd80fd550 2023-10-07T23:16:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.879+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00ca20 con 0x7f0dd80fd550 2023-10-07T23:16:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.879+0000 7f0ddecde700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd81108b0 con 0x7f0dd80fd550 2023-10-07T23:16:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.879+0000 7f0ddecde700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd8110d20 con 0x7f0dd80fd550 2023-10-07T23:16:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.881+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db80052b0 con 0x7f0dd80fd550 2023-10-07T23:16:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.885+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f0dcc0090d0 con 0x7f0dd80fd550 2023-10-07T23:16:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.888+0000 7f0dd57fa700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0dc809b7e0 0x7f0dc809dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.888+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f0dcc0c4c90 con 0x7f0dd80fd550 2023-10-07T23:16:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.889+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f0dcc0c5180 con 0x7f0dd80fd550 2023-10-07T23:16:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.889+0000 7f0dd7fff700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0dc809b7e0 0x7f0dc809dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:25.891+0000 7f0dd7fff700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0dc809b7e0 0x7f0dc809dc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40047d0 tx=0x7f0dc4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.156+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0db8005830 con 0x7f0dd80fd550 2023-10-07T23:16:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.157+0000 7f0dd57fa700 1 -- 172.21.15.67:0/1694693431 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dcc019070 con 0x7f0dd80fd550 2023-10-07T23:16:26.158 INFO:teuthology.orchestra.run.smithi067.stdout:197568495666 2023-10-07T23:16:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0dc809b7e0 msgr2=0x7f0dc809dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0dc809b7e0 0x7f0dc809dc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40047d0 tx=0x7f0dc4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 msgr2=0x7f0dd81128c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 0x7f0dd81128c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0041d0 tx=0x7f0dcc00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 shutdown_connections 2023-10-07T23:16:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dd80fd550 0x7f0dd81128c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0dc809b7e0 0x7f0dc809dc90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dd8103800 0x7f0dd810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 --2- 172.21.15.67:0/1694693431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dd8102e60 0x7f0dd810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.159+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 >> 172.21.15.67:0/1694693431 conn(0x7f0dd8074320 msgr2=0x7f0dd8103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.160+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 shutdown_connections 2023-10-07T23:16:26.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.160+0000 7f0dc2ffd700 1 -- 172.21.15.67:0/1694693431 wait complete. 2023-10-07T23:16:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 -- 172.21.15.67:0/2928915212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d8102e60 msgr2=0x7f15d8103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 --2- 172.21.15.67:0/2928915212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d8102e60 0x7f15d8103230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15c8004900 tx=0x7f15c8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 -- 172.21.15.67:0/2928915212 shutdown_connections 2023-10-07T23:16:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 --2- 172.21.15.67:0/2928915212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d8102e60 0x7f15d8103230 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15c8004900 tx=0x7f15c8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 --2- 172.21.15.67:0/2928915212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d80fd550 0x7f15d8109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 --2- 172.21.15.67:0/2928915212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d80fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.181+0000 7f15de6df700 1 -- 172.21.15.67:0/2928915212 >> 172.21.15.67:0/2928915212 conn(0x7f15d8074320 msgr2=0x7f15d8074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.183+0000 7f15de6df700 1 -- 172.21.15.67:0/2928915212 shutdown_connections 2023-10-07T23:16:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.183+0000 7f15de6df700 1 -- 172.21.15.67:0/2928915212 wait complete. 2023-10-07T23:16:26.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.184+0000 7f15de6df700 1 Processor -- start 2023-10-07T23:16:26.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.184+0000 7f15de6df700 1 -- start start 2023-10-07T23:16:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15de6df700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d80fd550 0x7f15d8113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15de6df700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d81141e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15de6df700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d810dd50 0x7f15d810e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:26.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15de6df700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d810e790 con 0x7f15d810dd50 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15de6df700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d810e900 con 0x7f15d80fd550 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15de6df700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d810ea70 con 0x7f15d8103800 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d81141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d80fd550 0x7f15d8113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d81141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:35760/0 (socket says 172.21.15.67:35760) 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.185+0000 7f15d77fe700 1 -- 172.21.15.67:0/2687099773 learned_addr learned my addr 172.21.15.67:0/2687099773 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d77fe700 1 -- 172.21.15.67:0/2687099773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d80fd550 msgr2=0x7f15d8113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d77fe700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d80fd550 0x7f15d8113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d77fe700 1 -- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d810dd50 msgr2=0x7f15d810e1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d77fe700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d810dd50 0x7f15d810e1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d77fe700 1 -- 172.21.15.67:0/2687099773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8005040 con 0x7f15d8103800 2023-10-07T23:16:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d77fe700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d81141e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00ebc0 tx=0x7f15cc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc007cc0 con 0x7f15d8103800 2023-10-07T23:16:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15cc00f040 con 0x7f15d8103800 2023-10-07T23:16:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d806fe70 con 0x7f15d8103800 2023-10-07T23:16:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.186+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00c990 con 0x7f15d8103800 2023-10-07T23:16:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.187+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d80703b0 con 0x7f15d8103800 2023-10-07T23:16:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.188+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8063f20 con 0x7f15d8103800 2023-10-07T23:16:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.190+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f15cc00cd40 con 0x7f15d8103800 2023-10-07T23:16:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.191+0000 7f15d4ff9700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f15c009b7f0 0x7f15c009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.191+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f15cc0c4bd0 con 0x7f15d8103800 2023-10-07T23:16:26.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.191+0000 7f15d7fff700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f15c009b7f0 0x7f15c009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:26.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.192+0000 7f15d7fff700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f15c009b7f0 0x7f15c009dca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15c8007f50 tx=0x7f15c8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.195+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f15cc04da00 con 0x7f15d8103800 2023-10-07T23:16:26.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.424+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f15d810f280 con 0x7f15d8103800 2023-10-07T23:16:26.438 INFO:teuthology.orchestra.run.smithi067.stdout:34359738448 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.425+0000 7f15d4ff9700 1 -- 172.21.15.67:0/2687099773 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15cc019020 con 0x7f15d8103800 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f15c009b7f0 msgr2=0x7f15c009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f15c009b7f0 0x7f15c009dca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15c8007f50 tx=0x7f15c8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 msgr2=0x7f15d81141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d81141e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00ebc0 tx=0x7f15cc00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 shutdown_connections 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d80fd550 0x7f15d8113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f15c009b7f0 0x7f15c009dca0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d810dd50 0x7f15d810e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 --2- 172.21.15.67:0/2687099773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d8103800 0x7f15d81141e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:26.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 >> 172.21.15.67:0/2687099773 conn(0x7f15d8074320 msgr2=0x7f15d80fb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:26.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 shutdown_connections 2023-10-07T23:16:26.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:26.427+0000 7f15de6df700 1 -- 172.21.15.67:0/2687099773 wait complete. 2023-10-07T23:16:27.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:26 smithi067 ceph-mon[82723]: pgmap v318: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:27.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1694693431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-07T23:16:27.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2687099773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-07T23:16:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:26 smithi123 ceph-mon[122926]: pgmap v318: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1694693431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-07T23:16:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2687099773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-07T23:16:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:26 smithi086 ceph-mon[122131]: pgmap v318: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1694693431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-07T23:16:27.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2687099773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-07T23:16:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 -- 172.21.15.67:0/190566635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 msgr2=0x7f3bb01051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 --2- 172.21.15.67:0/190566635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb01051a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40049c0 tx=0x7f3ba4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 -- 172.21.15.67:0/190566635 shutdown_connections 2023-10-07T23:16:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 --2- 172.21.15.67:0/190566635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb010ffc0 0x7f3bb01123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 --2- 172.21.15.67:0/190566635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb006f580 0x7f3bb0104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 --2- 172.21.15.67:0/190566635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb01051a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.508+0000 7f3bb546c700 1 -- 172.21.15.67:0/190566635 >> 172.21.15.67:0/190566635 conn(0x7f3bb006a3d0 msgr2=0x7f3bb006a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.520+0000 7f3bb546c700 1 -- 172.21.15.67:0/190566635 shutdown_connections 2023-10-07T23:16:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.520+0000 7f3bb546c700 1 -- 172.21.15.67:0/190566635 wait complete. 2023-10-07T23:16:27.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.521+0000 7f3bb546c700 1 Processor -- start 2023-10-07T23:16:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.521+0000 7f3bb546c700 1 -- start start 2023-10-07T23:16:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bb546c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb006f580 0x7f3bb010fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bb546c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb0108a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bb546c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb010ffc0 0x7f3bb0109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bb546c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb010d590 con 0x7f3bb006f9e0 2023-10-07T23:16:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bb546c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb010d700 con 0x7f3bb006f580 2023-10-07T23:16:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bb546c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb010d8c0 con 0x7f3bb010ffc0 2023-10-07T23:16:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bae7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb0108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bae7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb0108a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38600/0 (socket says 172.21.15.67:38600) 2023-10-07T23:16:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3bae7fc700 1 -- 172.21.15.67:0/2446232900 learned_addr learned my addr 172.21.15.67:0/2446232900 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3baf7fe700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb010ffc0 0x7f3bb0109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.522+0000 7f3baeffd700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb006f580 0x7f3bb010fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3baeffd700 1 -- 172.21.15.67:0/2446232900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb010ffc0 msgr2=0x7f3bb0109150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3baeffd700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb010ffc0 0x7f3bb0109150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3baeffd700 1 -- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 msgr2=0x7f3bb0108a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3baeffd700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb0108a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3baeffd700 1 -- 172.21.15.67:0/2446232900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4005040 con 0x7f3bb006f580 2023-10-07T23:16:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3bae7fc700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3baeffd700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb006f580 0x7f3bb010fa50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009fd0 tx=0x7f3ba000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.523+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00047d0 con 0x7f3bb006f580 2023-10-07T23:16:27.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.524+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb010db40 con 0x7f3bb006f580 2023-10-07T23:16:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.524+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb010e080 con 0x7f3bb006f580 2023-10-07T23:16:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.524+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba000f040 con 0x7f3bb006f580 2023-10-07T23:16:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.524+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000c8f0 con 0x7f3bb006f580 2023-10-07T23:16:27.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.526+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c0052e0 con 0x7f3bb006f580 2023-10-07T23:16:27.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.529+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f3ba000cd20 con 0x7f3bb006f580 2023-10-07T23:16:27.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7f3b977fe700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3b9809b860 0x7f3b9809dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.532+0000 7f3bae7fc700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3b9809b860 0x7f3b9809dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7f3bae7fc700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3b9809b860 0x7f3b9809dd10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3bb010a2b0 tx=0x7f3ba4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:27.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7f3ba00c4ae0 con 0x7f3bb006f580 2023-10-07T23:16:27.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f3ba00efa00 con 0x7f3bb006f580 2023-10-07T23:16:27.567 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495666 for osd.6 2023-10-07T23:16:27.568 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.713+0000 7fce7669c700 1 -- 172.21.15.67:0/3667074472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce70103800 msgr2=0x7fce700fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.713+0000 7fce7669c700 1 --2- 172.21.15.67:0/3667074472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce70103800 0x7fce700fce70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce640049f0 tx=0x7fce64009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 -- 172.21.15.67:0/3667074472 shutdown_connections 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 --2- 172.21.15.67:0/3667074472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce70102e60 0x7fce70103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 --2- 172.21.15.67:0/3667074472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce700fd550 0x7fce70109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 --2- 172.21.15.67:0/3667074472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce70103800 0x7fce700fce70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 -- 172.21.15.67:0/3667074472 >> 172.21.15.67:0/3667074472 conn(0x7fce70074320 msgr2=0x7fce70074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 -- 172.21.15.67:0/3667074472 shutdown_connections 2023-10-07T23:16:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.714+0000 7fce7669c700 1 -- 172.21.15.67:0/3667074472 wait complete. 2023-10-07T23:16:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.715+0000 7fce7669c700 1 Processor -- start 2023-10-07T23:16:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.715+0000 7fce7669c700 1 -- start start 2023-10-07T23:16:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.715+0000 7fce7669c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 0x7fce70115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.715+0000 7fce7669c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce70102e60 0x7fce70116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce7669c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce70103800 0x7fce7010fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce7669c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70110410 con 0x7fce700fd550 2023-10-07T23:16:27.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce7669c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70110580 con 0x7fce70102e60 2023-10-07T23:16:27.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce7669c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70110740 con 0x7fce70103800 2023-10-07T23:16:27.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 0x7fce70115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 0x7fce70115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38630/0 (socket says 172.21.15.67:38630) 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce6ffff700 1 -- 172.21.15.67:0/3230796199 learned_addr learned my addr 172.21.15.67:0/3230796199 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce74c39700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce70103800 0x7fce7010fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.716+0000 7fce6f7fe700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce70102e60 0x7fce70116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6ffff700 1 -- 172.21.15.67:0/3230796199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce70103800 msgr2=0x7fce7010fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6ffff700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce70103800 0x7fce7010fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6ffff700 1 -- 172.21.15.67:0/3230796199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce70102e60 msgr2=0x7fce70116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6ffff700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce70102e60 0x7fce70116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6ffff700 1 -- 172.21.15.67:0/3230796199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64005040 con 0x7fce700fd550 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce74c39700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce70103800 0x7fce7010fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:16:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6ffff700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 0x7fce70115d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce60002ad0 tx=0x7fce600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60007cc0 con 0x7fce700fd550 2023-10-07T23:16:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6000f040 con 0x7fce700fd550 2023-10-07T23:16:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000da00 con 0x7fce700fd550 2023-10-07T23:16:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.717+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce70110a20 con 0x7fce700fd550 2023-10-07T23:16:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.718+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7011cd30 con 0x7fce700fd550 2023-10-07T23:16:27.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.719+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce70063f20 con 0x7fce700fd550 2023-10-07T23:16:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.722+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fce600090d0 con 0x7fce700fd550 2023-10-07T23:16:27.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.724+0000 7fce6cff9700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fce5809b7f0 0x7fce5809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:27.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.724+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7fce600c5990 con 0x7fce700fd550 2023-10-07T23:16:27.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.724+0000 7fce6f7fe700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fce5809b7f0 0x7fce5809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:27.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.725+0000 7fce6f7fe700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fce5809b7f0 0x7fce5809dca0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce64004890 tx=0x7fce64009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:27.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.727+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fce60090f60 con 0x7fce700fd550 2023-10-07T23:16:27.815 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.814+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f3b9c005860 con 0x7f3bb006f580 2023-10-07T23:16:27.815 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.815+0000 7f3b977fe700 1 -- 172.21.15.67:0/2446232900 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f3ba0019020 con 0x7f3bb006f580 2023-10-07T23:16:27.816 INFO:teuthology.orchestra.run.smithi067.stdout:309237645340 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3b9809b860 msgr2=0x7f3b9809dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3b9809b860 0x7f3b9809dd10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3bb010a2b0 tx=0x7f3ba4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb006f580 msgr2=0x7f3bb010fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb006f580 0x7f3bb010fa50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009fd0 tx=0x7f3ba000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 shutdown_connections 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb006f580 0x7f3bb010fa50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3b9809b860 0x7f3b9809dd10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb010ffc0 0x7f3bb0109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 --2- 172.21.15.67:0/2446232900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb006f9e0 0x7f3bb0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 >> 172.21.15.67:0/2446232900 conn(0x7f3bb006a3d0 msgr2=0x7f3bb0113a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:27.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 shutdown_connections 2023-10-07T23:16:27.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.818+0000 7f3bb546c700 1 -- 172.21.15.67:0/2446232900 wait complete. 2023-10-07T23:16:27.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.962+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fce7011cec0 con 0x7fce700fd550 2023-10-07T23:16:27.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.964+0000 7fce6cff9700 1 -- 172.21.15.67:0/3230796199 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce6000ddf0 con 0x7fce700fd550 2023-10-07T23:16:27.966 INFO:teuthology.orchestra.run.smithi067.stdout:55834574923 2023-10-07T23:16:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fce5809b7f0 msgr2=0x7fce5809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fce5809b7f0 0x7fce5809dca0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce64004890 tx=0x7fce64009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 msgr2=0x7fce70115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 0x7fce70115d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce60002ad0 tx=0x7fce600046b0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 shutdown_connections 2023-10-07T23:16:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fce5809b7f0 0x7fce5809dca0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce64004890 tx=0x7fce64009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce70103800 0x7fce7010fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce700fd550 0x7fce70115d40 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce60002ad0 tx=0x7fce600046b0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 --2- 172.21.15.67:0/3230796199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce70102e60 0x7fce70116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.967+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 >> 172.21.15.67:0/3230796199 conn(0x7fce70074320 msgr2=0x7fce7006fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.969+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 shutdown_connections 2023-10-07T23:16:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.969+0000 7fce7669c700 1 -- 172.21.15.67:0/3230796199 wait complete. 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/918206596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2446232900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-07T23:16:28.083 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-07T23:16:28.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-10-07T23:16:28.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2023-10-07T23:16:28.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-10-07T23:16:28.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2023-10-07T23:16:28.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-10-07T23:16:28.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3230796199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-07T23:16:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 -- 172.21.15.67:0/3823478794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 msgr2=0x7ff62c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 --2- 172.21.15.67:0/3823478794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c103230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0049f0 tx=0x7ff61c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 -- 172.21.15.67:0/3823478794 shutdown_connections 2023-10-07T23:16:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 --2- 172.21.15.67:0/3823478794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff62c0fd550 0x7ff62c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 --2- 172.21.15.67:0/3823478794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c103230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 --2- 172.21.15.67:0/3823478794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 0x7ff62c0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 -- 172.21.15.67:0/3823478794 >> 172.21.15.67:0/3823478794 conn(0x7ff62c074320 msgr2=0x7ff62c074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 -- 172.21.15.67:0/3823478794 shutdown_connections 2023-10-07T23:16:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.128+0000 7ff63154c700 1 -- 172.21.15.67:0/3823478794 wait complete. 2023-10-07T23:16:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.129+0000 7ff63154c700 1 Processor -- start 2023-10-07T23:16:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.129+0000 7ff63154c700 1 -- start start 2023-10-07T23:16:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff63154c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff62c0fd550 0x7ff62c113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff63154c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff63154c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 0x7ff62c10daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff63154c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c10e140 con 0x7ff62c102e60 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff63154c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c10e2b0 con 0x7ff62c103800 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff63154c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c10e470 con 0x7ff62c0fd550 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff62a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff62a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38652/0 (socket says 172.21.15.67:38652) 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff62a7fc700 1 -- 172.21.15.67:0/3676851167 learned_addr learned my addr 172.21.15.67:0/3676851167 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff62affd700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff62c0fd550 0x7ff62c113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.130+0000 7ff62b7fe700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 0x7ff62c10daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62a7fc700 1 -- 172.21.15.67:0/3676851167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff62c0fd550 msgr2=0x7ff62c113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62a7fc700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff62c0fd550 0x7ff62c113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62a7fc700 1 -- 172.21.15.67:0/3676851167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 msgr2=0x7ff62c10daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62a7fc700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 0x7ff62c10daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62a7fc700 1 -- 172.21.15.67:0/3676851167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c005040 con 0x7ff62c102e60 2023-10-07T23:16:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62b7fe700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 0x7ff62c10daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff62a7fc700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c113f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff62000ebc0 tx=0x7ff62000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620007cc0 con 0x7ff62c102e60 2023-10-07T23:16:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff62000f040 con 0x7ff62c102e60 2023-10-07T23:16:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000c990 con 0x7ff62c102e60 2023-10-07T23:16:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.131+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c10e6f0 con 0x7ff62c102e60 2023-10-07T23:16:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.132+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c11aa30 con 0x7ff62c102e60 2023-10-07T23:16:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.133+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c063f20 con 0x7ff62c102e60 2023-10-07T23:16:28.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.135+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7ff62000cd40 con 0x7ff62c102e60 2023-10-07T23:16:28.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.136+0000 7ff6137fe700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff61409b770 0x7ff61409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.136+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9016+0+0 (secure 0 0 0) 0x7ff6200c4c00 con 0x7ff62c102e60 2023-10-07T23:16:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.136+0000 7ff62affd700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff61409b770 0x7ff61409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.137+0000 7ff62affd700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff61409b770 0x7ff61409dc20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009fd0 tx=0x7ff61c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.140+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7ff6200900b0 con 0x7ff62c102e60 2023-10-07T23:16:28.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.363+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7ff62c11ac80 con 0x7ff62c102e60 2023-10-07T23:16:28.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.364+0000 7ff6137fe700 1 -- 172.21.15.67:0/3676851167 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff620019090 con 0x7ff62c102e60 2023-10-07T23:16:28.365 INFO:teuthology.orchestra.run.smithi067.stdout:253403070504 2023-10-07T23:16:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff61409b770 msgr2=0x7ff61409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff61409b770 0x7ff61409dc20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009fd0 tx=0x7ff61c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 msgr2=0x7ff62c113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c113f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff62000ebc0 tx=0x7ff62000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 shutdown_connections 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff62c103800 0x7ff62c10daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff61409b770 0x7ff61409dc20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff62c0fd550 0x7ff62c113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.366+0000 7ff63154c700 1 --2- 172.21.15.67:0/3676851167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c102e60 0x7ff62c113f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.367+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 >> 172.21.15.67:0/3676851167 conn(0x7ff62c074320 msgr2=0x7ff62c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.367+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 shutdown_connections 2023-10-07T23:16:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:28.367+0000 7ff63154c700 1 -- 172.21.15.67:0/3676851167 wait complete. 2023-10-07T23:16:28.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/918206596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-07T23:16:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2446232900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-07T23:16:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-07T23:16:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-07T23:16:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-10-07T23:16:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-07T23:16:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-07T23:16:28.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-10-07T23:16:28.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2023-10-07T23:16:28.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-10-07T23:16:28.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2023-10-07T23:16:28.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-10-07T23:16:28.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3230796199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-07T23:16:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/918206596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-07T23:16:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2446232900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-07T23:16:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2023-10-07T23:16:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-10-07T23:16:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3230796199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-07T23:16:28.830 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738448 for osd.0 2023-10-07T23:16:28.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.527+0000 7ff8114fd700 1 -- 172.21.15.67:0/3049095327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c102e60 msgr2=0x7ff80c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.527+0000 7ff8114fd700 1 --2- 172.21.15.67:0/3049095327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c102e60 0x7ff80c103230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8000049c0 tx=0x7ff800009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.527+0000 7ff8114fd700 1 -- 172.21.15.67:0/3049095327 shutdown_connections 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.527+0000 7ff8114fd700 1 --2- 172.21.15.67:0/3049095327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c102e60 0x7ff80c103230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.527+0000 7ff8114fd700 1 --2- 172.21.15.67:0/3049095327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff80c0fd550 0x7ff80c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.527+0000 7ff8114fd700 1 --2- 172.21.15.67:0/3049095327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c103800 0x7ff80c0fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.528+0000 7ff8114fd700 1 -- 172.21.15.67:0/3049095327 >> 172.21.15.67:0/3049095327 conn(0x7ff80c074320 msgr2=0x7ff80c074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.528+0000 7ff8114fd700 1 -- 172.21.15.67:0/3049095327 shutdown_connections 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.529+0000 7ff8114fd700 1 -- 172.21.15.67:0/3049095327 wait complete. 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.530+0000 7ff8114fd700 1 Processor -- start 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.530+0000 7ff8114fd700 1 -- start start 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff8114fd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff80c0fd550 0x7ff80c1058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff8114fd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 0x7ff80c105e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff8114fd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c103800 0x7ff80c118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff8114fd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c119400 con 0x7ff80c0fd550 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff8114fd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c119570 con 0x7ff80c103800 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff8114fd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c119730 con 0x7ff80c102e60 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.531+0000 7ff80a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 0x7ff80c105e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.532+0000 7ff80a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 0x7ff80c105e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:35792/0 (socket says 172.21.15.67:35792) 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.532+0000 7ff80a7fc700 1 -- 172.21.15.67:0/918206596 learned_addr learned my addr 172.21.15.67:0/918206596 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.533+0000 7ff80a7fc700 1 -- 172.21.15.67:0/918206596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c103800 msgr2=0x7ff80c118dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.533+0000 7ff80a7fc700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c103800 0x7ff80c118dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.533+0000 7ff80a7fc700 1 -- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff80c0fd550 msgr2=0x7ff80c1058d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.533+0000 7ff80a7fc700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff80c0fd550 0x7ff80c1058d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.533+0000 7ff80a7fc700 1 -- 172.21.15.67:0/918206596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800005040 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.533+0000 7ff80a7fc700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 0x7ff80c105e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00ebc0 tx=0x7ff7fc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007cc0 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7fc00f040 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00ca10 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c1199b0 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.534+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c1210f0 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.536+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c063f20 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.540+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7ff7fc0090d0 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.541+0000 7ff7f37fe700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff7f409b7f0 0x7ff7f409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.541+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8726+0+0 (secure 0 0 0) 0x7ff7fc0c5470 con 0x7ff80c102e60 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.542+0000 7ff80affd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff7f409b7f0 0x7ff7f409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:28.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.542+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7ff7fc090a40 con 0x7ff80c102e60 2023-10-07T23:16:28.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.542+0000 7ff80affd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff7f409b7f0 0x7ff7f409dca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff800005010 tx=0x7ff800009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:28.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.779+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ff80c0709a0 con 0x7ff80c102e60 2023-10-07T23:16:28.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.780+0000 7ff7f37fe700 1 -- 172.21.15.67:0/918206596 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7fc019070 con 0x7ff80c102e60 2023-10-07T23:16:28.834 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:28.834 INFO:teuthology.orchestra.run.smithi067.stdout:287762808866 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff7f409b7f0 msgr2=0x7ff7f409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff7f409b7f0 0x7ff7f409dca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff800005010 tx=0x7ff800009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 msgr2=0x7ff80c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 0x7ff80c105e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00ebc0 tx=0x7ff7fc00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 shutdown_connections 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff80c103800 0x7ff80c118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ff7f409b7f0 0x7ff7f409dca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff80c0fd550 0x7ff80c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 --2- 172.21.15.67:0/918206596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102e60 0x7ff80c105e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 >> 172.21.15.67:0/918206596 conn(0x7ff80c074320 msgr2=0x7ff80c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 shutdown_connections 2023-10-07T23:16:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:27.784+0000 7ff8114fd700 1 -- 172.21.15.67:0/918206596 wait complete. 2023-10-07T23:16:29.224 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645338 got 309237645340 for osd.10 2023-10-07T23:16:29.224 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: pgmap v319: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2023-10-07T23:16:29.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-10-07T23:16:29.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2023-10-07T23:16:29.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-10-07T23:16:29.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: osdmap e90: 12 total, 12 up, 12 in 2023-10-07T23:16:29.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:16:29.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:29 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3676851167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-07T23:16:29.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: pgmap v319: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2023-10-07T23:16:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-10-07T23:16:29.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: osdmap e90: 12 total, 12 up, 12 in 2023-10-07T23:16:29.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:16:29.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:29 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3676851167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-07T23:16:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: pgmap v319: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2023-10-07T23:16:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2023-10-07T23:16:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: osdmap e90: 12 total, 12 up, 12 in 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-07T23:16:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:29 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3676851167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-07T23:16:29.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 -- 172.21.15.67:0/3142128801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c06ff80 msgr2=0x7fd34c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:29.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 --2- 172.21.15.67:0/3142128801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c06ff80 0x7fd34c070350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0049c0 tx=0x7fd33c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:29.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 -- 172.21.15.67:0/3142128801 shutdown_connections 2023-10-07T23:16:29.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 --2- 172.21.15.67:0/3142128801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c06ff80 0x7fd34c070350 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 --2- 172.21.15.67:0/3142128801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c109710 0x7fd34c10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 --2- 172.21.15.67:0/3142128801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c070890 0x7fd34c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.019+0000 7fd35278b700 1 -- 172.21.15.67:0/3142128801 >> 172.21.15.67:0/3142128801 conn(0x7fd34c074320 msgr2=0x7fd34c074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.020+0000 7fd35278b700 1 -- 172.21.15.67:0/3142128801 shutdown_connections 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.020+0000 7fd35278b700 1 -- 172.21.15.67:0/3142128801 wait complete. 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.020+0000 7fd35278b700 1 Processor -- start 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.020+0000 7fd35278b700 1 -- start start 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd35278b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 0x7fd34c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd35278b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c070890 0x7fd34c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd35278b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c109710 0x7fd34c10fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd35278b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c1103d0 con 0x7fd34c06ff80 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd35278b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c110540 con 0x7fd34c070890 2023-10-07T23:16:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd35278b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c110700 con 0x7fd34c109710 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd34bfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 0x7fd34c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd34b7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c070890 0x7fd34c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd34bfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 0x7fd34c115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38680/0 (socket says 172.21.15.67:38680) 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd34bfff700 1 -- 172.21.15.67:0/3846320338 learned_addr learned my addr 172.21.15.67:0/3846320338 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd350d28700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c109710 0x7fd34c10fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.021+0000 7fd34bfff700 1 -- 172.21.15.67:0/3846320338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c109710 msgr2=0x7fd34c10fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd34bfff700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c109710 0x7fd34c10fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd34bfff700 1 -- 172.21.15.67:0/3846320338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c070890 msgr2=0x7fd34c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd34bfff700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c070890 0x7fd34c116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd34bfff700 1 -- 172.21.15.67:0/3846320338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c005040 con 0x7fd34c06ff80 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd34b7fe700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c070890 0x7fd34c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd34bfff700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 0x7fd34c115d00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009b70 tx=0x7fd33c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00e040 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c110980 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33c0097c0 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c013e00 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.022+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c11ccf0 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.023+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c063f20 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.025+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd33c0134b0 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.027+0000 7fd348ff9700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd33409b7f0 0x7fd33409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.027+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fd33c0c5da0 con 0x7fd34c06ff80 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.028+0000 7fd34b7fe700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd33409b7f0 0x7fd33409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.028+0000 7fd34b7fe700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd33409b7f0 0x7fd33409dca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1115a0 tx=0x7fd34000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:29.458 INFO:teuthology.orchestra.run.smithi067.stdout:77309411397 2023-10-07T23:16:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.032+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd33c091250 con 0x7fd34c06ff80 2023-10-07T23:16:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.246+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd34c10eef0 con 0x7fd34c06ff80 2023-10-07T23:16:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.246+0000 7fd348ff9700 1 -- 172.21.15.67:0/3846320338 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd33c021090 con 0x7fd34c06ff80 2023-10-07T23:16:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd33409b7f0 msgr2=0x7fd33409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd33409b7f0 0x7fd33409dca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1115a0 tx=0x7fd34000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 msgr2=0x7fd34c115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 0x7fd34c115d00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009b70 tx=0x7fd33c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 shutdown_connections 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd33409b7f0 0x7fd33409dca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd34c109710 0x7fd34c10fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd34c06ff80 0x7fd34c115d00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 --2- 172.21.15.67:0/3846320338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34c070890 0x7fd34c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 >> 172.21.15.67:0/3846320338 conn(0x7fd34c074320 msgr2=0x7fd34c103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 shutdown_connections 2023-10-07T23:16:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:29.249+0000 7fd35278b700 1 -- 172.21.15.67:0/3846320338 wait complete. 2023-10-07T23:16:29.474 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070503 got 253403070504 for osd.8 2023-10-07T23:16:29.475 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:29.919 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574923 for osd.1 2023-10-07T23:16:29.919 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:30.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:30 smithi067 ceph-mon[82723]: osdmap e91: 12 total, 12 up, 12 in 2023-10-07T23:16:30.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:30 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3846320338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-07T23:16:30.293 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411397 for osd.2 2023-10-07T23:16:30.293 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:30.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:30 smithi123 ceph-mon[122926]: osdmap e91: 12 total, 12 up, 12 in 2023-10-07T23:16:30.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:30 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3846320338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-07T23:16:30.442 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:30.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:30 smithi086 ceph-mon[122131]: osdmap e91: 12 total, 12 up, 12 in 2023-10-07T23:16:30.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:30 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3846320338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-07T23:16:30.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808866 for osd.9 2023-10-07T23:16:30.491 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 -- 172.21.15.67:0/4151285844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26940fbe10 msgr2=0x7f26940fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 --2- 172.21.15.67:0/4151285844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26940fbe10 0x7f26940fc280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2684004980 tx=0x7f2684009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 -- 172.21.15.67:0/4151285844 shutdown_connections 2023-10-07T23:16:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 --2- 172.21.15.67:0/4151285844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2694101d70 0x7f2694102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 --2- 172.21.15.67:0/4151285844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f2694108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 --2- 172.21.15.67:0/4151285844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26940fbe10 0x7f26940fc280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 -- 172.21.15.67:0/4151285844 >> 172.21.15.67:0/4151285844 conn(0x7f2694071c80 msgr2=0x7f2694072080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 -- 172.21.15.67:0/4151285844 shutdown_connections 2023-10-07T23:16:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.023+0000 7f269c693700 1 -- 172.21.15.67:0/4151285844 wait complete. 2023-10-07T23:16:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.024+0000 7f269c693700 1 Processor -- start 2023-10-07T23:16:31.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.024+0000 7f269c693700 1 -- start start 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269c693700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26940fbe10 0x7f269410b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269c693700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f269410bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269c693700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2694101d70 0x7f2694128a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269c693700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269410c780 con 0x7f26940fc7c0 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269c693700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694128fc0 con 0x7f26940fbe10 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269c693700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694129100 con 0x7f2694101d70 2023-10-07T23:16:31.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f269410bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269a42f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26940fbe10 0x7f269410b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f269ac30700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2694101d70 0x7f2694128a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f269410bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38710/0 (socket says 172.21.15.67:38710) 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 -- 172.21.15.67:0/3509971549 learned_addr learned my addr 172.21.15.67:0/3509971549 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 -- 172.21.15.67:0/3509971549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2694101d70 msgr2=0x7f2694128a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2694101d70 0x7f2694128a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 -- 172.21.15.67:0/3509971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26940fbe10 msgr2=0x7f269410b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26940fbe10 0x7f269410b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.025+0000 7f2699c2e700 1 -- 172.21.15.67:0/3509971549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684005040 con 0x7f26940fc7c0 2023-10-07T23:16:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.026+0000 7f2699c2e700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f269410bec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2684009fd0 tx=0x7f2684009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.026+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684011480 con 0x7f26940fc7c0 2023-10-07T23:16:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.026+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2694129380 con 0x7f26940fc7c0 2023-10-07T23:16:31.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.026+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2684009930 con 0x7f26940fc7c0 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.026+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400f680 con 0x7f26940fc7c0 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.026+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2694129830 con 0x7f26940fc7c0 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.027+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694063f20 con 0x7f26940fc7c0 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.029+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f268401a980 con 0x7f26940fc7c0 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.031+0000 7f268affd700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f268009b7f0 0x7f268009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.031+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f26840c69b0 con 0x7f26940fc7c0 2023-10-07T23:16:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.032+0000 7f269a42f700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f268009b7f0 0x7f268009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.033+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2684092000 con 0x7f26940fc7c0 2023-10-07T23:16:31.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.034+0000 7f269a42f700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f268009b7f0 0x7f268009dca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2690009b80 tx=0x7f269000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:31 smithi067 ceph-mon[82723]: pgmap v322: 129 pgs: 5 peering, 124 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:31 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.255+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f269410aaf0 con 0x7f26940fc7c0 2023-10-07T23:16:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.256+0000 7f268affd700 1 -- 172.21.15.67:0/3509971549 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f268401e020 con 0x7f26940fc7c0 2023-10-07T23:16:31.258 INFO:teuthology.orchestra.run.smithi067.stdout:107374182465 2023-10-07T23:16:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f268009b7f0 msgr2=0x7f268009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f268009b7f0 0x7f268009dca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2690009b80 tx=0x7f269000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:31.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 msgr2=0x7f269410bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f269410bec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2684009fd0 tx=0x7f2684009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:31.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 shutdown_connections 2023-10-07T23:16:31.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26940fbe10 0x7f269410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f268009b7f0 0x7f268009dca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2694101d70 0x7f2694128a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 --2- 172.21.15.67:0/3509971549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26940fc7c0 0x7f269410bec0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.258+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 >> 172.21.15.67:0/3509971549 conn(0x7f2694071c80 msgr2=0x7f26940fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:31.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.259+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 shutdown_connections 2023-10-07T23:16:31.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.259+0000 7f269c693700 1 -- 172.21.15.67:0/3509971549 wait complete. 2023-10-07T23:16:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:31 smithi123 ceph-mon[122926]: pgmap v322: 129 pgs: 5 peering, 124 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:31 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:31 smithi086 ceph-mon[122131]: pgmap v322: 129 pgs: 5 peering, 124 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:31 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.742+0000 7fe162a56700 1 -- 172.21.15.67:0/148275116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c0fd3f0 msgr2=0x7fe15c109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.742+0000 7fe162a56700 1 --2- 172.21.15.67:0/148275116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c109ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe154004d10 comp rx=0 tx=0).stop 2023-10-07T23:16:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 -- 172.21.15.67:0/148275116 shutdown_connections 2023-10-07T23:16:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 --2- 172.21.15.67:0/148275116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c102d00 0x7fe15c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 --2- 172.21.15.67:0/148275116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c109ae0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 --2- 172.21.15.67:0/148275116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe15c1036a0 0x7fe15c0fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 -- 172.21.15.67:0/148275116 >> 172.21.15.67:0/148275116 conn(0x7fe15c0f8870 msgr2=0x7fe15c0fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:31.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 -- 172.21.15.67:0/148275116 shutdown_connections 2023-10-07T23:16:31.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 -- 172.21.15.67:0/148275116 wait complete. 2023-10-07T23:16:31.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.743+0000 7fe162a56700 1 Processor -- start 2023-10-07T23:16:31.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 -- start start 2023-10-07T23:16:31.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c115a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe15c102d00 0x7fe15c115f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c1036a0 0x7fe15c10fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c110120 con 0x7fe15c1036a0 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c110290 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe162a56700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c110450 con 0x7fe15c102d00 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe161a54700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c115a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe161a54700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c115a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:34060/0 (socket says 172.21.15.67:34060) 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe161a54700 1 -- 172.21.15.67:0/617348469 learned_addr learned my addr 172.21.15.67:0/617348469 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe162255700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c1036a0 0x7fe15c10fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.744+0000 7fe161253700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe15c102d00 0x7fe15c115f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe161a54700 1 -- 172.21.15.67:0/617348469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe15c102d00 msgr2=0x7fe15c115f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe161a54700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe15c102d00 0x7fe15c115f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe161a54700 1 -- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c1036a0 msgr2=0x7fe15c10fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe161a54700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c1036a0 0x7fe15c10fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe161a54700 1 -- 172.21.15.67:0/617348469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154009940 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe162255700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c1036a0 0x7fe15c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe161a54700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c115a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe15800ebc0 tx=0x7fe15800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158007cc0 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe15800f040 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580094a0 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.745+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c1106d0 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.746+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c11ca10 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.746+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c04c3c0 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.748+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fe15800c600 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.750+0000 7fe1527fc700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe14809b7f0 0x7fe14809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:31.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.750+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fe1580c4e30 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.750+0000 7fe161253700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe14809b7f0 0x7fe14809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:31.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.751+0000 7fe161253700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe14809b7f0 0x7fe14809dca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe15c071ed0 tx=0x7fe14c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.753+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fe15804da00 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.990+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fe15c11cba0 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.991+0000 7fe1527fc700 1 -- 172.21.15.67:0/617348469 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe158019020 con 0x7fe15c0fd3f0 2023-10-07T23:16:31.991 INFO:teuthology.orchestra.run.smithi067.stdout:343597383704 2023-10-07T23:16:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe14809b7f0 msgr2=0x7fe14809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe14809b7f0 0x7fe14809dca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe15c071ed0 tx=0x7fe14c00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 msgr2=0x7fe15c115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c115a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe15800ebc0 tx=0x7fe15800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 shutdown_connections 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0fd3f0 0x7fe15c115a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe14809b7f0 0x7fe14809dca0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe15c1036a0 0x7fe15c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 --2- 172.21.15.67:0/617348469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe15c102d00 0x7fe15c115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 >> 172.21.15.67:0/617348469 conn(0x7fe15c0f8870 msgr2=0x7fe15c107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 shutdown_connections 2023-10-07T23:16:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:31.993+0000 7fe162a56700 1 -- 172.21.15.67:0/617348469 wait complete. 2023-10-07T23:16:32.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3509971549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-07T23:16:32.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/617348469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-07T23:16:32.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3509971549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-07T23:16:32.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/617348469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-07T23:16:32.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3509971549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-07T23:16:32.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/617348469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-07T23:16:32.727 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182465 for osd.3 2023-10-07T23:16:32.727 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 -- 172.21.15.67:0/3199212102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 msgr2=0x7f5828105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 --2- 172.21.15.67:0/3199212102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828105430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f58180049f0 tx=0x7f5818009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 -- 172.21.15.67:0/3199212102 shutdown_connections 2023-10-07T23:16:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 --2- 172.21.15.67:0/3199212102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f582810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 --2- 172.21.15.67:0/3199212102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828105430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 --2- 172.21.15.67:0/3199212102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5828105a00 0x7f58280ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 -- 172.21.15.67:0/3199212102 >> 172.21.15.67:0/3199212102 conn(0x7f58280fabf0 msgr2=0x7f58280fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 -- 172.21.15.67:0/3199212102 shutdown_connections 2023-10-07T23:16:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.827+0000 7f582c994700 1 -- 172.21.15.67:0/3199212102 wait complete. 2023-10-07T23:16:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.828+0000 7f582c994700 1 Processor -- start 2023-10-07T23:16:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.828+0000 7f582c994700 1 -- start start 2023-10-07T23:16:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.828+0000 7f582c994700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f5828115ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.828+0000 7f582c994700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828116400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.828+0000 7f582c994700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5828105a00 0x7f582810ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582c994700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828110590 con 0x7f5828105060 2023-10-07T23:16:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582c994700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828110700 con 0x7f5828105a00 2023-10-07T23:16:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582c994700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281108c0 con 0x7f58280ff750 2023-10-07T23:16:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f5828115ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f5825d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828116400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f5828115ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:49388/0 (socket says 172.21.15.67:49388) 2023-10-07T23:16:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 -- 172.21.15.67:0/3796492699 learned_addr learned my addr 172.21.15.67:0/3796492699 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f5826d9d700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5828105a00 0x7f582810ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 -- 172.21.15.67:0/3796492699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5828105a00 msgr2=0x7f582810ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5828105a00 0x7f582810ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 -- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 msgr2=0x7f5828116400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828116400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f582659c700 1 -- 172.21.15.67:0/3796492699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818005040 con 0x7f58280ff750 2023-10-07T23:16:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.829+0000 7f5825d9b700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828116400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.830+0000 7f582659c700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f5828115ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5818009400 tx=0x7f5818009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.830+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581800e070 con 0x7f58280ff750 2023-10-07T23:16:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.830+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5828110b40 con 0x7f58280ff750 2023-10-07T23:16:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.830+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582811ceb0 con 0x7f58280ff750 2023-10-07T23:16:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.830+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5818004180 con 0x7f58280ff750 2023-10-07T23:16:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.830+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581801c430 con 0x7f58280ff750 2023-10-07T23:16:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.831+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5828063f20 con 0x7f58280ff750 2023-10-07T23:16:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.833+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f581801c860 con 0x7f58280ff750 2023-10-07T23:16:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.834+0000 7f5816ffd700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f581009b770 0x7f581009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.835+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f58180c6a80 con 0x7f58280ff750 2023-10-07T23:16:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.835+0000 7f5825d9b700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f581009b770 0x7f581009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.835+0000 7f5825d9b700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f581009b770 0x7f581009dc20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5828111760 tx=0x7f581c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:32.836+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f5818092040 con 0x7f58280ff750 2023-10-07T23:16:33.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.064+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f58281114d0 con 0x7f58280ff750 2023-10-07T23:16:33.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.064+0000 7f5816ffd700 1 -- 172.21.15.67:0/3796492699 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5818095660 con 0x7f58280ff750 2023-10-07T23:16:33.065 INFO:teuthology.orchestra.run.smithi067.stdout:141733920828 2023-10-07T23:16:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f581009b770 msgr2=0x7f581009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f581009b770 0x7f581009dc20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5828111760 tx=0x7f581c00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 msgr2=0x7f5828115ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f5828115ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5818009400 tx=0x7f5818009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 shutdown_connections 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5828105a00 0x7f582810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f581009b770 0x7f581009dc20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58280ff750 0x7f5828115ec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 --2- 172.21.15.67:0/3796492699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5828105060 0x7f5828116400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 >> 172.21.15.67:0/3796492699 conn(0x7f58280fabf0 msgr2=0x7f58280fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 shutdown_connections 2023-10-07T23:16:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.067+0000 7f582c994700 1 -- 172.21.15.67:0/3796492699 wait complete. 2023-10-07T23:16:33.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383704 for osd.11 2023-10-07T23:16:33.072 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:33.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:33 smithi067 ceph-mon[82723]: pgmap v323: 129 pgs: 5 peering, 124 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:33.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:33 smithi123 ceph-mon[122926]: pgmap v323: 129 pgs: 5 peering, 124 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:33.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:33 smithi086 ceph-mon[122131]: pgmap v323: 129 pgs: 5 peering, 124 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 -- 172.21.15.67:0/665065350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105a00 msgr2=0x7f9a380ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 --2- 172.21.15.67:0/665065350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105a00 0x7f9a380ff070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a280049f0 tx=0x7f9a28009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 -- 172.21.15.67:0/665065350 shutdown_connections 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 --2- 172.21.15.67:0/665065350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a38105060 0x7f9a38105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 --2- 172.21.15.67:0/665065350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a380ff750 0x7f9a3810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 --2- 172.21.15.67:0/665065350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105a00 0x7f9a380ff070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 -- 172.21.15.67:0/665065350 >> 172.21.15.67:0/665065350 conn(0x7f9a380fabf0 msgr2=0x7f9a380fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 -- 172.21.15.67:0/665065350 shutdown_connections 2023-10-07T23:16:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.491+0000 7f9a404df700 1 -- 172.21.15.67:0/665065350 wait complete. 2023-10-07T23:16:33.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 Processor -- start 2023-10-07T23:16:33.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 -- start start 2023-10-07T23:16:33.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a380ff750 0x7f9a38115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 0x7f9a381161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a38105a00 0x7f9a3810fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a38110360 con 0x7f9a38105060 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381104d0 con 0x7f9a380ff750 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.492+0000 7f9a404df700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a38110690 con 0x7f9a38105a00 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3da7a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 0x7f9a381161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3ea7c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a38105a00 0x7f9a3810fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a380ff750 0x7f9a38115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3da7a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 0x7f9a381161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48640/0 (socket says 172.21.15.67:48640) 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3da7a700 1 -- 172.21.15.67:0/3197663349 learned_addr learned my addr 172.21.15.67:0/3197663349 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3ea7c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a38105a00 0x7f9a3810fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:49400/0 (socket says 172.21.15.67:49400) 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 -- 172.21.15.67:0/3197663349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a38105a00 msgr2=0x7f9a3810fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a38105a00 0x7f9a3810fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 -- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 msgr2=0x7f9a381161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 0x7f9a381161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 -- 172.21.15.67:0/3197663349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28005040 con 0x7f9a380ff750 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3da7a700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 0x7f9a381161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.493+0000 7f9a3e27b700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a380ff750 0x7f9a38115c90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009fd0 tx=0x7f9a3400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.494+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34007cc0 con 0x7f9a380ff750 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.494+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a38110910 con 0x7f9a380ff750 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.494+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a3400f040 con 0x7f9a380ff750 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.494+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400c980 con 0x7f9a380ff750 2023-10-07T23:16:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.494+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3811cc80 con 0x7f9a380ff750 2023-10-07T23:16:33.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.495+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a38063f20 con 0x7f9a380ff750 2023-10-07T23:16:33.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.497+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9a340090d0 con 0x7f9a380ff750 2023-10-07T23:16:33.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.498+0000 7f9a2effd700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a2409b770 0x7f9a2409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.498+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f9a340c4c90 con 0x7f9a380ff750 2023-10-07T23:16:33.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.498+0000 7f9a3da7a700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a2409b770 0x7f9a2409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.499+0000 7f9a3da7a700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a2409b770 0x7f9a2409dc20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a38111530 tx=0x7f9a28009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:33.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.502+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9a3404da00 con 0x7f9a380ff750 2023-10-07T23:16:33.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 -- 172.21.15.67:0/4228428407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec101d20 msgr2=0x7f1eec065630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 --2- 172.21.15.67:0/4228428407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec101d20 0x7f1eec065630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80049c0 tx=0x7f1ee8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:33.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 -- 172.21.15.67:0/4228428407 shutdown_connections 2023-10-07T23:16:33.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 --2- 172.21.15.67:0/4228428407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec0ff840 0x7f1eec0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 --2- 172.21.15.67:0/4228428407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec101d20 0x7f1eec065630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 --2- 172.21.15.67:0/4228428407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1eec065b70 0x7f1eec065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.609+0000 7f1ef354c700 1 -- 172.21.15.67:0/4228428407 >> 172.21.15.67:0/4228428407 conn(0x7f1eec071c80 msgr2=0x7f1eec072080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.610+0000 7f1ef354c700 1 -- 172.21.15.67:0/4228428407 shutdown_connections 2023-10-07T23:16:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.610+0000 7f1ef354c700 1 -- 172.21.15.67:0/4228428407 wait complete. 2023-10-07T23:16:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.611+0000 7f1ef354c700 1 Processor -- start 2023-10-07T23:16:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.611+0000 7f1ef354c700 1 -- start start 2023-10-07T23:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.611+0000 7f1ef354c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1eec065b70 0x7f1eec11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.611+0000 7f1ef354c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 0x7f1eec11a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.611+0000 7f1ef354c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 0x7f1eec114680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef354c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec114c50 con 0x7f1eec0ff840 2023-10-07T23:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef354c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec114dc0 con 0x7f1eec114210 2023-10-07T23:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef354c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec114f30 con 0x7f1eec065b70 2023-10-07T23:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef0ae7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 0x7f1eec11a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef1ae9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 0x7f1eec114680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef12e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1eec065b70 0x7f1eec11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef1ae9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 0x7f1eec114680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:55760/0 (socket says 172.21.15.67:55760) 2023-10-07T23:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef0ae7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 0x7f1eec11a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48654/0 (socket says 172.21.15.67:48654) 2023-10-07T23:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef1ae9700 1 -- 172.21.15.67:0/1632476145 learned_addr learned my addr 172.21.15.67:0/1632476145 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef0ae7700 1 -- 172.21.15.67:0/1632476145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1eec065b70 msgr2=0x7f1eec11a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.612+0000 7f1ef0ae7700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1eec065b70 0x7f1eec11a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ef0ae7700 1 -- 172.21.15.67:0/1632476145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 msgr2=0x7f1eec114680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ef0ae7700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 0x7f1eec114680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ef0ae7700 1 -- 172.21.15.67:0/1632476145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8005040 con 0x7f1eec0ff840 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ef1ae9700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 0x7f1eec114680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ef0ae7700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 0x7f1eec11a6a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009fd0 tx=0x7f1edc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0047d0 con 0x7f1eec0ff840 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1edc00f040 con 0x7f1eec0ff840 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.613+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00c820 con 0x7f1eec0ff840 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.614+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec120f90 con 0x7f1eec0ff840 2023-10-07T23:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.615+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec1212e0 con 0x7f1eec0ff840 2023-10-07T23:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.616+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec063f20 con 0x7f1eec0ff840 2023-10-07T23:16:33.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.620+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f1edc0092e0 con 0x7f1eec0ff840 2023-10-07T23:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.623+0000 7f1ee1ffb700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1ed809b7f0 0x7f1ed809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.623+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f1edc0c48a0 con 0x7f1eec0ff840 2023-10-07T23:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.624+0000 7f1ef12e8700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1ed809b7f0 0x7f1ed809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.625+0000 7f1ef12e8700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1ed809b7f0 0x7f1ed809dca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8013460 tx=0x7f1ee8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:33.628 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.627+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f1edc08fdb0 con 0x7f1eec0ff840 2023-10-07T23:16:33.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920828 for osd.4 2023-10-07T23:16:33.646 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:33.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.751+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9a3811ce10 con 0x7f9a380ff750 2023-10-07T23:16:33.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.752+0000 7f9a2effd700 1 -- 172.21.15.67:0/3197663349 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a34019020 con 0x7f9a380ff750 2023-10-07T23:16:33.753 INFO:teuthology.orchestra.run.smithi067.stdout:219043332140 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a2409b770 msgr2=0x7f9a2409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a2409b770 0x7f9a2409dc20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a38111530 tx=0x7f9a28009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a380ff750 msgr2=0x7f9a38115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a380ff750 0x7f9a38115c90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009fd0 tx=0x7f9a3400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 shutdown_connections 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a380ff750 0x7f9a38115c90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9a2409b770 0x7f9a2409dc20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a38105a00 0x7f9a3810fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 --2- 172.21.15.67:0/3197663349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38105060 0x7f9a381161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 >> 172.21.15.67:0/3197663349 conn(0x7f9a380fabf0 msgr2=0x7f9a380fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:33.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 shutdown_connections 2023-10-07T23:16:33.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.756+0000 7f9a404df700 1 -- 172.21.15.67:0/3197663349 wait complete. 2023-10-07T23:16:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.850+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1eec1131b0 con 0x7f1eec0ff840 2023-10-07T23:16:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.851+0000 7f1ee1ffb700 1 -- 172.21.15.67:0/1632476145 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1edc019090 con 0x7f1eec0ff840 2023-10-07T23:16:33.851 INFO:teuthology.orchestra.run.smithi067.stdout:163208757304 2023-10-07T23:16:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.853+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1ed809b7f0 msgr2=0x7f1ed809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.853+0000 7f1ef354c700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1ed809b7f0 0x7f1ed809dca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8013460 tx=0x7f1ee8009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.853+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 msgr2=0x7f1eec11a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.853+0000 7f1ef354c700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 0x7f1eec11a6a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009fd0 tx=0x7f1edc00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 shutdown_connections 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1eec114210 0x7f1eec114680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1ed809b7f0 0x7f1ed809dca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1eec065b70 0x7f1eec11a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 --2- 172.21.15.67:0/1632476145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eec0ff840 0x7f1eec11a6a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 >> 172.21.15.67:0/1632476145 conn(0x7f1eec071c80 msgr2=0x7f1eec0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 shutdown_connections 2023-10-07T23:16:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:33.854+0000 7f1ef354c700 1 -- 172.21.15.67:0/1632476145 wait complete. 2023-10-07T23:16:34.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:34 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3796492699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-07T23:16:34.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:34 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3197663349' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-07T23:16:34.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:34 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1632476145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-07T23:16:34.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:34 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3796492699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-07T23:16:34.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:34 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3197663349' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-07T23:16:34.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:34 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1632476145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-07T23:16:34.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:34 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3796492699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-07T23:16:34.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:34 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3197663349' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-07T23:16:34.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:34 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1632476145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-07T23:16:34.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332140 for osd.7 2023-10-07T23:16:34.516 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:34.517 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757304 for osd.5 2023-10-07T23:16:34.517 DEBUG:teuthology.parallel:result is None 2023-10-07T23:16:34.517 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-07T23:16:34.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-07T23:16:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:35.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:35 smithi123 ceph-mon[122926]: pgmap v324: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:35.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:35 smithi086 ceph-mon[122131]: pgmap v324: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:35.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:35 smithi067 ceph-mon[82723]: pgmap v324: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 -- 172.21.15.67:0/1636375606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c103800 msgr2=0x7fb75c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1636375606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c103800 0x7fb75c0fce70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb74c004930 tx=0x7fb74c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 -- 172.21.15.67:0/1636375606 shutdown_connections 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1636375606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1636375606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb75c102e60 0x7fb75c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1636375606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c103800 0x7fb75c0fce70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 -- 172.21.15.67:0/1636375606 >> 172.21.15.67:0/1636375606 conn(0x7fb75c074320 msgr2=0x7fb75c074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 -- 172.21.15.67:0/1636375606 shutdown_connections 2023-10-07T23:16:36.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.800+0000 7fb7637b0700 1 -- 172.21.15.67:0/1636375606 wait complete. 2023-10-07T23:16:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 Processor -- start 2023-10-07T23:16:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 -- start start 2023-10-07T23:16:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c102e60 0x7fb75c116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb75c103800 0x7fb75c10fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:36.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c110400 con 0x7fb75c103800 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c110570 con 0x7fb75c0fd550 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb7637b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c110730 con 0x7fb75c102e60 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.801+0000 7fb76154c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:55776/0 (socket says 172.21.15.67:55776) 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb760d4b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c102e60 0x7fb75c116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 -- 172.21.15.67:0/1582495276 learned_addr learned my addr 172.21.15.67:0/1582495276 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb761d4d700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb75c103800 0x7fb75c10fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 -- 172.21.15.67:0/1582495276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c102e60 msgr2=0x7fb75c116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c102e60 0x7fb75c116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 -- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb75c103800 msgr2=0x7fb75c10fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:36.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb75c103800 0x7fb75c10fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 -- 172.21.15.67:0/1582495276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758005040 con 0x7fb75c0fd550 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb76154c700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c115d30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb758009730 tx=0x7fb758009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.802+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758007be0 con 0x7fb75c0fd550 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.803+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c005040 con 0x7fb75c0fd550 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.803+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75800f040 con 0x7fb75c0fd550 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.803+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758013cd0 con 0x7fb75c0fd550 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.803+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c11cb60 con 0x7fb75c0fd550 2023-10-07T23:16:36.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.804+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c063f20 con 0x7fb75c0fd550 2023-10-07T23:16:36.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.806+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb75800e840 con 0x7fb75c0fd550 2023-10-07T23:16:36.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.807+0000 7fb751ffb700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb74809b770 0x7fb74809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:36.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.807+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb7580c50e0 con 0x7fb75c0fd550 2023-10-07T23:16:36.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.807+0000 7fb760d4b700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb74809b770 0x7fb74809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:36.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.808+0000 7fb760d4b700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb74809b770 0x7fb74809dc20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009fd0 tx=0x7fb74c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:36.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:36.811+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fb758090590 con 0x7fb75c0fd550 2023-10-07T23:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.012+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb75c071590 con 0x7fb74809b770 2023-10-07T23:16:37.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.026+0000 7fb751ffb700 1 -- 172.21.15.67:0/1582495276 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426877 (secure 0 0 0) 0x7fb75c071590 con 0x7fb74809b770 2023-10-07T23:16:37.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:37.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.033+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb74809b770 msgr2=0x7fb74809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:37.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.033+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb74809b770 0x7fb74809dc20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009fd0 tx=0x7fb74c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:37.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.033+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 msgr2=0x7fb75c115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:37.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.033+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c115d30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb758009730 tx=0x7fb758009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:37.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 shutdown_connections 2023-10-07T23:16:37.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb75c0fd550 0x7fb75c115d30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb74809b770 0x7fb74809dc20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb75c103800 0x7fb75c10fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 --2- 172.21.15.67:0/1582495276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb75c102e60 0x7fb75c116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 >> 172.21.15.67:0/1582495276 conn(0x7fb75c074320 msgr2=0x7fb75c06fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 shutdown_connections 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:37.034+0000 7fb7637b0700 1 -- 172.21.15.67:0/1582495276 wait complete. 2023-10-07T23:16:37.039 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:16:37.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:37 smithi123 ceph-mon[122926]: pgmap v325: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:37.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:37.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:37 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:37.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:37 smithi067 ceph-mon[82723]: pgmap v325: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:37.408 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:37.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:37 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:37 smithi086 ceph-mon[122131]: pgmap v325: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:37.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:37 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:37.455 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2023-10-07T23:16:35.938475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600173,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":544,"num_read_kb":336,"num_write":390,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3581412,"kb_used_data":13956,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121360412,"statfs":{"total":1151940427776,"available":1148273061888,"internally_reserved":0,"allocated":14290944,"data_stored":7729521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001990"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004139+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:16:28.004139+0000","last_peered":"2023-10-07T23:16:28.004139+0000","last_clean":"2023-10-07T23:16:28.004139+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:16:28.004139+0000","last_undegraded":"2023-10-07T23:16:28.004139+0000","last_fullsized":"2023-10-07T23:16:28.004139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012736+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:16:28.012736+0000","last_peered":"2023-10-07T23:16:28.012736+0000","last_clean":"2023-10-07T23:16:28.012736+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:16:28.012736+0000","last_undegraded":"2023-10-07T23:16:28.012736+0000","last_fullsized":"2023-10-07T23:16:28.012736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012507+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:16:28.012507+0000","last_peered":"2023-10-07T23:16:28.012507+0000","last_clean":"2023-10-07T23:16:28.012507+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:16:28.012507+0000","last_undegraded":"2023-10-07T23:16:28.012507+0000","last_fullsized":"2023-10-07T23:16:28.012507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999764+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:16:27.999764+0000","last_peered":"2023-10-07T23:16:27.999764+0000","last_clean":"2023-10-07T23:16:27.999764+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:16:27.999764+0000","last_undegraded":"2023-10-07T23:16:27.999764+0000","last_fullsized":"2023-10-07T23:16:27.999764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011235+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:16:28.011235+0000","last_peered":"2023-10-07T23:16:28.011235+0000","last_clean":"2023-10-07T23:16:28.011235+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:16:28.011235+0000","last_undegraded":"2023-10-07T23:16:28.011235+0000","last_fullsized":"2023-10-07T23:16:28.011235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003946+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:16:28.003946+0000","last_peered":"2023-10-07T23:16:28.003946+0000","last_clean":"2023-10-07T23:16:28.003946+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:16:28.003946+0000","last_undegraded":"2023-10-07T23:16:28.003946+0000","last_fullsized":"2023-10-07T23:16:28.003946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007260+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:16:28.007260+0000","last_peered":"2023-10-07T23:16:28.007260+0000","last_clean":"2023-10-07T23:16:28.007260+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:16:28.007260+0000","last_undegraded":"2023-10-07T23:16:28.007260+0000","last_fullsized":"2023-10-07T23:16:28.007260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015359+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:16:28.015359+0000","last_peered":"2023-10-07T23:16:28.015359+0000","last_clean":"2023-10-07T23:16:28.015359+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:16:28.015359+0000","last_undegraded":"2023-10-07T23:16:28.015359+0000","last_fullsized":"2023-10-07T23:16:28.015359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006337+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:16:28.006337+0000","last_peered":"2023-10-07T23:16:28.006337+0000","last_clean":"2023-10-07T23:16:28.006337+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:16:28.006337+0000","last_undegraded":"2023-10-07T23:16:28.006337+0000","last_fullsized":"2023-10-07T23:16:28.006337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003351+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:16:28.003351+0000","last_peered":"2023-10-07T23:16:28.003351+0000","last_clean":"2023-10-07T23:16:28.003351+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:16:28.003351+0000","last_undegraded":"2023-10-07T23:16:28.003351+0000","last_fullsized":"2023-10-07T23:16:28.003351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007883+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:16:28.007883+0000","last_peered":"2023-10-07T23:16:28.007883+0000","last_clean":"2023-10-07T23:16:28.007883+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:16:28.007883+0000","last_undegraded":"2023-10-07T23:16:28.007883+0000","last_fullsized":"2023-10-07T23:16:28.007883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999630+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:16:27.999630+0000","last_peered":"2023-10-07T23:16:27.999630+0000","last_clean":"2023-10-07T23:16:27.999630+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:16:27.999630+0000","last_undegraded":"2023-10-07T23:16:27.999630+0000","last_fullsized":"2023-10-07T23:16:27.999630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000842+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:16:28.000842+0000","last_peered":"2023-10-07T23:16:28.000842+0000","last_clean":"2023-10-07T23:16:28.000842+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:16:28.000842+0000","last_undegraded":"2023-10-07T23:16:28.000842+0000","last_fullsized":"2023-10-07T23:16:28.000842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017901+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:16:28.017901+0000","last_peered":"2023-10-07T23:16:28.017901+0000","last_clean":"2023-10-07T23:16:28.017901+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:16:28.017901+0000","last_undegraded":"2023-10-07T23:16:28.017901+0000","last_fullsized":"2023-10-07T23:16:28.017901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011942+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:16:28.011942+0000","last_peered":"2023-10-07T23:16:28.011942+0000","last_clean":"2023-10-07T23:16:28.011942+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:16:28.011942+0000","last_undegraded":"2023-10-07T23:16:28.011942+0000","last_fullsized":"2023-10-07T23:16:28.011942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005454+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:16:28.005454+0000","last_peered":"2023-10-07T23:16:28.005454+0000","last_clean":"2023-10-07T23:16:28.005454+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:16:28.005454+0000","last_undegraded":"2023-10-07T23:16:28.005454+0000","last_fullsized":"2023-10-07T23:16:28.005454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004354+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:16:28.004354+0000","last_peered":"2023-10-07T23:16:28.004354+0000","last_clean":"2023-10-07T23:16:28.004354+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:16:28.004354+0000","last_undegraded":"2023-10-07T23:16:28.004354+0000","last_fullsized":"2023-10-07T23:16:28.004354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009520+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:16:28.009520+0000","last_peered":"2023-10-07T23:16:28.009520+0000","last_clean":"2023-10-07T23:16:28.009520+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.009520+0000","last_undegraded":"2023-10-07T23:16:28.009520+0000","last_fullsized":"2023-10-07T23:16:28.009520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008489+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:16:28.008489+0000","last_peered":"2023-10-07T23:16:28.008489+0000","last_clean":"2023-10-07T23:16:28.008489+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:16:28.008489+0000","last_undegraded":"2023-10-07T23:16:28.008489+0000","last_fullsized":"2023-10-07T23:16:28.008489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006630+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:16:28.006630+0000","last_peered":"2023-10-07T23:16:28.006630+0000","last_clean":"2023-10-07T23:16:28.006630+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:16:28.006630+0000","last_undegraded":"2023-10-07T23:16:28.006630+0000","last_fullsized":"2023-10-07T23:16:28.006630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:31.914321+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:16:31.914321+0000","last_peered":"2023-10-07T23:16:31.914321+0000","last_clean":"2023-10-07T23:16:31.914321+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:16:31.914321+0000","last_undegraded":"2023-10-07T23:16:31.914321+0000","last_fullsized":"2023-10-07T23:16:31.914321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014500+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:16:28.014500+0000","last_peered":"2023-10-07T23:16:28.014500+0000","last_clean":"2023-10-07T23:16:28.014500+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:16:28.014500+0000","last_undegraded":"2023-10-07T23:16:28.014500+0000","last_fullsized":"2023-10-07T23:16:28.014500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999223+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:16:27.999223+0000","last_peered":"2023-10-07T23:16:27.999223+0000","last_clean":"2023-10-07T23:16:27.999223+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:16:27.999223+0000","last_undegraded":"2023-10-07T23:16:27.999223+0000","last_fullsized":"2023-10-07T23:16:27.999223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014597+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:16:28.014597+0000","last_peered":"2023-10-07T23:16:28.014597+0000","last_clean":"2023-10-07T23:16:28.014597+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:16:28.014597+0000","last_undegraded":"2023-10-07T23:16:28.014597+0000","last_fullsized":"2023-10-07T23:16:28.014597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012905+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:16:28.012905+0000","last_peered":"2023-10-07T23:16:28.012905+0000","last_clean":"2023-10-07T23:16:28.012905+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:16:28.012905+0000","last_undegraded":"2023-10-07T23:16:28.012905+0000","last_fullsized":"2023-10-07T23:16:28.012905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999373+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:16:27.999373+0000","last_peered":"2023-10-07T23:16:27.999373+0000","last_clean":"2023-10-07T23:16:27.999373+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:16:27.999373+0000","last_undegraded":"2023-10-07T23:16:27.999373+0000","last_fullsized":"2023-10-07T23:16:27.999373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010037+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:16:28.010037+0000","last_peered":"2023-10-07T23:16:28.010037+0000","last_clean":"2023-10-07T23:16:28.010037+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:16:28.010037+0000","last_undegraded":"2023-10-07T23:16:28.010037+0000","last_fullsized":"2023-10-07T23:16:28.010037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005917+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:16:28.005917+0000","last_peered":"2023-10-07T23:16:28.005917+0000","last_clean":"2023-10-07T23:16:28.005917+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:16:28.005917+0000","last_undegraded":"2023-10-07T23:16:28.005917+0000","last_fullsized":"2023-10-07T23:16:28.005917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999220+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:16:27.999220+0000","last_peered":"2023-10-07T23:16:27.999220+0000","last_clean":"2023-10-07T23:16:27.999220+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:16:27.999220+0000","last_undegraded":"2023-10-07T23:16:27.999220+0000","last_fullsized":"2023-10-07T23:16:27.999220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011048+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:16:28.011048+0000","last_peered":"2023-10-07T23:16:28.011048+0000","last_clean":"2023-10-07T23:16:28.011048+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:16:28.011048+0000","last_undegraded":"2023-10-07T23:16:28.011048+0000","last_fullsized":"2023-10-07T23:16:28.011048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005339+0000","last_change":"2023-10-07T23:15:27.197818+0000","last_active":"2023-10-07T23:16:28.005339+0000","last_peered":"2023-10-07T23:16:28.005339+0000","last_clean":"2023-10-07T23:16:28.005339+0000","last_became_active":"2023-10-07T23:15:27.197522+0000","last_became_peered":"2023-10-07T23:15:27.197522+0000","last_unstale":"2023-10-07T23:16:28.005339+0000","last_undegraded":"2023-10-07T23:16:28.005339+0000","last_fullsized":"2023-10-07T23:16:28.005339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:11.128855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008264+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:16:28.008264+0000","last_peered":"2023-10-07T23:16:28.008264+0000","last_clean":"2023-10-07T23:16:28.008264+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:16:28.008264+0000","last_undegraded":"2023-10-07T23:16:28.008264+0000","last_fullsized":"2023-10-07T23:16:28.008264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:30.894641+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:16:30.894641+0000","last_peered":"2023-10-07T23:16:30.894641+0000","last_clean":"2023-10-07T23:16:30.894641+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:16:30.894641+0000","last_undegraded":"2023-10-07T23:16:30.894641+0000","last_fullsized":"2023-10-07T23:16:30.894641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010625+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:16:28.010625+0000","last_peered":"2023-10-07T23:16:28.010625+0000","last_clean":"2023-10-07T23:16:28.010625+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:16:28.010625+0000","last_undegraded":"2023-10-07T23:16:28.010625+0000","last_fullsized":"2023-10-07T23:16:28.010625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.013091+0000","last_change":"2023-10-07T23:16:29.013091+0000","last_active":"2023-10-07T23:16:29.013091+0000","last_peered":"2023-10-07T23:16:29.013091+0000","last_clean":"2023-10-07T23:16:29.013091+0000","last_became_active":"2023-10-07T23:16:29.012680+0000","last_became_peered":"2023-10-07T23:16:29.012680+0000","last_unstale":"2023-10-07T23:16:29.013091+0000","last_undegraded":"2023-10-07T23:16:29.013091+0000","last_fullsized":"2023-10-07T23:16:29.013091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:13.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.050161+0000","last_change":"2023-10-07T23:16:29.050161+0000","last_active":"2023-10-07T23:16:29.050161+0000","last_peered":"2023-10-07T23:16:29.050161+0000","last_clean":"2023-10-07T23:16:29.050161+0000","last_became_active":"2023-10-07T23:16:29.044559+0000","last_became_peered":"2023-10-07T23:16:29.044559+0000","last_unstale":"2023-10-07T23:16:29.050161+0000","last_undegraded":"2023-10-07T23:16:29.050161+0000","last_fullsized":"2023-10-07T23:16:29.050161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:18.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006444+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:16:28.006444+0000","last_peered":"2023-10-07T23:16:28.006444+0000","last_clean":"2023-10-07T23:16:28.006444+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:16:28.006444+0000","last_undegraded":"2023-10-07T23:16:28.006444+0000","last_fullsized":"2023-10-07T23:16:28.006444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045657+0000","last_change":"2023-10-07T23:16:29.045657+0000","last_active":"2023-10-07T23:16:29.045657+0000","last_peered":"2023-10-07T23:16:29.045657+0000","last_clean":"2023-10-07T23:16:29.045657+0000","last_became_active":"2023-10-07T23:16:29.045268+0000","last_became_peered":"2023-10-07T23:16:29.045268+0000","last_unstale":"2023-10-07T23:16:29.045657+0000","last_undegraded":"2023-10-07T23:16:29.045657+0000","last_fullsized":"2023-10-07T23:16:29.045657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:34.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999559+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:16:27.999559+0000","last_peered":"2023-10-07T23:16:27.999559+0000","last_clean":"2023-10-07T23:16:27.999559+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:16:27.999559+0000","last_undegraded":"2023-10-07T23:16:27.999559+0000","last_fullsized":"2023-10-07T23:16:27.999559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016282+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:16:28.016282+0000","last_peered":"2023-10-07T23:16:28.016282+0000","last_clean":"2023-10-07T23:16:28.016282+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:16:28.016282+0000","last_undegraded":"2023-10-07T23:16:28.016282+0000","last_fullsized":"2023-10-07T23:16:28.016282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012663+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:16:28.012663+0000","last_peered":"2023-10-07T23:16:28.012663+0000","last_clean":"2023-10-07T23:16:28.012663+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:16:28.012663+0000","last_undegraded":"2023-10-07T23:16:28.012663+0000","last_fullsized":"2023-10-07T23:16:28.012663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003839+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:16:28.003839+0000","last_peered":"2023-10-07T23:16:28.003839+0000","last_clean":"2023-10-07T23:16:28.003839+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:16:28.003839+0000","last_undegraded":"2023-10-07T23:16:28.003839+0000","last_fullsized":"2023-10-07T23:16:28.003839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005155+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:16:28.005155+0000","last_peered":"2023-10-07T23:16:28.005155+0000","last_clean":"2023-10-07T23:16:28.005155+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:16:28.005155+0000","last_undegraded":"2023-10-07T23:16:28.005155+0000","last_fullsized":"2023-10-07T23:16:28.005155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008218+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:16:28.008218+0000","last_peered":"2023-10-07T23:16:28.008218+0000","last_clean":"2023-10-07T23:16:28.008218+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:16:28.008218+0000","last_undegraded":"2023-10-07T23:16:28.008218+0000","last_fullsized":"2023-10-07T23:16:28.008218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:31.895250+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:16:31.895250+0000","last_peered":"2023-10-07T23:16:31.895250+0000","last_clean":"2023-10-07T23:16:31.895250+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:16:31.895250+0000","last_undegraded":"2023-10-07T23:16:31.895250+0000","last_fullsized":"2023-10-07T23:16:31.895250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006164+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:16:28.006164+0000","last_peered":"2023-10-07T23:16:28.006164+0000","last_clean":"2023-10-07T23:16:28.006164+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:16:28.006164+0000","last_undegraded":"2023-10-07T23:16:28.006164+0000","last_fullsized":"2023-10-07T23:16:28.006164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004871+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:16:28.004871+0000","last_peered":"2023-10-07T23:16:28.004871+0000","last_clean":"2023-10-07T23:16:28.004871+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:16:28.004871+0000","last_undegraded":"2023-10-07T23:16:28.004871+0000","last_fullsized":"2023-10-07T23:16:28.004871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007922+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:16:28.007922+0000","last_peered":"2023-10-07T23:16:28.007922+0000","last_clean":"2023-10-07T23:16:28.007922+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:16:28.007922+0000","last_undegraded":"2023-10-07T23:16:28.007922+0000","last_fullsized":"2023-10-07T23:16:28.007922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017142+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:16:28.017142+0000","last_peered":"2023-10-07T23:16:28.017142+0000","last_clean":"2023-10-07T23:16:28.017142+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:16:28.017142+0000","last_undegraded":"2023-10-07T23:16:28.017142+0000","last_fullsized":"2023-10-07T23:16:28.017142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999614+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:16:27.999614+0000","last_peered":"2023-10-07T23:16:27.999614+0000","last_clean":"2023-10-07T23:16:27.999614+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:16:27.999614+0000","last_undegraded":"2023-10-07T23:16:27.999614+0000","last_fullsized":"2023-10-07T23:16:27.999614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005267+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:16:28.005267+0000","last_peered":"2023-10-07T23:16:28.005267+0000","last_clean":"2023-10-07T23:16:28.005267+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:16:28.005267+0000","last_undegraded":"2023-10-07T23:16:28.005267+0000","last_fullsized":"2023-10-07T23:16:28.005267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006062+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:16:28.006062+0000","last_peered":"2023-10-07T23:16:28.006062+0000","last_clean":"2023-10-07T23:16:28.006062+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:16:28.006062+0000","last_undegraded":"2023-10-07T23:16:28.006062+0000","last_fullsized":"2023-10-07T23:16:28.006062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005402+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:16:28.005402+0000","last_peered":"2023-10-07T23:16:28.005402+0000","last_clean":"2023-10-07T23:16:28.005402+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:16:28.005402+0000","last_undegraded":"2023-10-07T23:16:28.005402+0000","last_fullsized":"2023-10-07T23:16:28.005402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'14","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:31.895276+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:16:31.895276+0000","last_peered":"2023-10-07T23:16:31.895276+0000","last_clean":"2023-10-07T23:16:31.895276+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:16:31.895276+0000","last_undegraded":"2023-10-07T23:16:31.895276+0000","last_fullsized":"2023-10-07T23:16:31.895276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":43,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009919+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:16:28.009919+0000","last_peered":"2023-10-07T23:16:28.009919+0000","last_clean":"2023-10-07T23:16:28.009919+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:16:28.009919+0000","last_undegraded":"2023-10-07T23:16:28.009919+0000","last_fullsized":"2023-10-07T23:16:28.009919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005504+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:16:28.005504+0000","last_peered":"2023-10-07T23:16:28.005504+0000","last_clean":"2023-10-07T23:16:28.005504+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:16:28.005504+0000","last_undegraded":"2023-10-07T23:16:28.005504+0000","last_fullsized":"2023-10-07T23:16:28.005504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007745+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:16:28.007745+0000","last_peered":"2023-10-07T23:16:28.007745+0000","last_clean":"2023-10-07T23:16:28.007745+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:16:28.007745+0000","last_undegraded":"2023-10-07T23:16:28.007745+0000","last_fullsized":"2023-10-07T23:16:28.007745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999913+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:16:27.999913+0000","last_peered":"2023-10-07T23:16:27.999913+0000","last_clean":"2023-10-07T23:16:27.999913+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:16:27.999913+0000","last_undegraded":"2023-10-07T23:16:27.999913+0000","last_fullsized":"2023-10-07T23:16:27.999913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002941+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:16:28.002941+0000","last_peered":"2023-10-07T23:16:28.002941+0000","last_clean":"2023-10-07T23:16:28.002941+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:16:28.002941+0000","last_undegraded":"2023-10-07T23:16:28.002941+0000","last_fullsized":"2023-10-07T23:16:28.002941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005850+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:16:28.005850+0000","last_peered":"2023-10-07T23:16:28.005850+0000","last_clean":"2023-10-07T23:16:28.005850+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:16:28.005850+0000","last_undegraded":"2023-10-07T23:16:28.005850+0000","last_fullsized":"2023-10-07T23:16:28.005850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008325+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:16:28.008325+0000","last_peered":"2023-10-07T23:16:28.008325+0000","last_clean":"2023-10-07T23:16:28.008325+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:16:28.008325+0000","last_undegraded":"2023-10-07T23:16:28.008325+0000","last_fullsized":"2023-10-07T23:16:28.008325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044573+0000","last_change":"2023-10-07T23:16:29.044573+0000","last_active":"2023-10-07T23:16:29.044573+0000","last_peered":"2023-10-07T23:16:29.044573+0000","last_clean":"2023-10-07T23:16:29.044573+0000","last_became_active":"2023-10-07T23:16:29.044228+0000","last_became_peered":"2023-10-07T23:16:29.044228+0000","last_unstale":"2023-10-07T23:16:29.044573+0000","last_undegraded":"2023-10-07T23:16:29.044573+0000","last_fullsized":"2023-10-07T23:16:29.044573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:31.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:30.894049+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:16:30.894049+0000","last_peered":"2023-10-07T23:16:30.894049+0000","last_clean":"2023-10-07T23:16:30.894049+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:16:30.894049+0000","last_undegraded":"2023-10-07T23:16:30.894049+0000","last_fullsized":"2023-10-07T23:16:30.894049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007188+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:16:28.007188+0000","last_peered":"2023-10-07T23:16:28.007188+0000","last_clean":"2023-10-07T23:16:28.007188+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:16:28.007188+0000","last_undegraded":"2023-10-07T23:16:28.007188+0000","last_fullsized":"2023-10-07T23:16:28.007188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003129+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:16:28.003129+0000","last_peered":"2023-10-07T23:16:28.003129+0000","last_clean":"2023-10-07T23:16:28.003129+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:16:28.003129+0000","last_undegraded":"2023-10-07T23:16:28.003129+0000","last_fullsized":"2023-10-07T23:16:28.003129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012034+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:16:28.012034+0000","last_peered":"2023-10-07T23:16:28.012034+0000","last_clean":"2023-10-07T23:16:28.012034+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:16:28.012034+0000","last_undegraded":"2023-10-07T23:16:28.012034+0000","last_fullsized":"2023-10-07T23:16:28.012034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003744+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:16:28.003744+0000","last_peered":"2023-10-07T23:16:28.003744+0000","last_clean":"2023-10-07T23:16:28.003744+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:16:28.003744+0000","last_undegraded":"2023-10-07T23:16:28.003744+0000","last_fullsized":"2023-10-07T23:16:28.003744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003691+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:16:28.003691+0000","last_peered":"2023-10-07T23:16:28.003691+0000","last_clean":"2023-10-07T23:16:28.003691+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:16:28.003691+0000","last_undegraded":"2023-10-07T23:16:28.003691+0000","last_fullsized":"2023-10-07T23:16:28.003691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012511+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:16:28.012511+0000","last_peered":"2023-10-07T23:16:28.012511+0000","last_clean":"2023-10-07T23:16:28.012511+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:16:28.012511+0000","last_undegraded":"2023-10-07T23:16:28.012511+0000","last_fullsized":"2023-10-07T23:16:28.012511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003423+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:16:28.003423+0000","last_peered":"2023-10-07T23:16:28.003423+0000","last_clean":"2023-10-07T23:16:28.003423+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:16:28.003423+0000","last_undegraded":"2023-10-07T23:16:28.003423+0000","last_fullsized":"2023-10-07T23:16:28.003423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010581+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:16:28.010581+0000","last_peered":"2023-10-07T23:16:28.010581+0000","last_clean":"2023-10-07T23:16:28.010581+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:16:28.010581+0000","last_undegraded":"2023-10-07T23:16:28.010581+0000","last_fullsized":"2023-10-07T23:16:28.010581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006472+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:16:28.006472+0000","last_peered":"2023-10-07T23:16:28.006472+0000","last_clean":"2023-10-07T23:16:28.006472+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:16:28.006472+0000","last_undegraded":"2023-10-07T23:16:28.006472+0000","last_fullsized":"2023-10-07T23:16:28.006472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003225+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:16:28.003225+0000","last_peered":"2023-10-07T23:16:28.003225+0000","last_clean":"2023-10-07T23:16:28.003225+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:16:28.003225+0000","last_undegraded":"2023-10-07T23:16:28.003225+0000","last_fullsized":"2023-10-07T23:16:28.003225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001591+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:16:28.001591+0000","last_peered":"2023-10-07T23:16:28.001591+0000","last_clean":"2023-10-07T23:16:28.001591+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:16:28.001591+0000","last_undegraded":"2023-10-07T23:16:28.001591+0000","last_fullsized":"2023-10-07T23:16:28.001591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:30.913695+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:16:30.913695+0000","last_peered":"2023-10-07T23:16:30.913695+0000","last_clean":"2023-10-07T23:16:30.913695+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:16:30.913695+0000","last_undegraded":"2023-10-07T23:16:30.913695+0000","last_fullsized":"2023-10-07T23:16:30.913695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003060+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:16:28.003060+0000","last_peered":"2023-10-07T23:16:28.003060+0000","last_clean":"2023-10-07T23:16:28.003060+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:16:28.003060+0000","last_undegraded":"2023-10-07T23:16:28.003060+0000","last_fullsized":"2023-10-07T23:16:28.003060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999866+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:16:27.999866+0000","last_peered":"2023-10-07T23:16:27.999866+0000","last_clean":"2023-10-07T23:16:27.999866+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:16:27.999866+0000","last_undegraded":"2023-10-07T23:16:27.999866+0000","last_fullsized":"2023-10-07T23:16:27.999866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004005+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:16:28.004005+0000","last_peered":"2023-10-07T23:16:28.004005+0000","last_clean":"2023-10-07T23:16:28.004005+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:16:28.004005+0000","last_undegraded":"2023-10-07T23:16:28.004005+0000","last_fullsized":"2023-10-07T23:16:28.004005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:31.894890+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:16:31.894890+0000","last_peered":"2023-10-07T23:16:31.894890+0000","last_clean":"2023-10-07T23:16:31.894890+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:16:31.894890+0000","last_undegraded":"2023-10-07T23:16:31.894890+0000","last_fullsized":"2023-10-07T23:16:31.894890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007777+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:16:28.007777+0000","last_peered":"2023-10-07T23:16:28.007777+0000","last_clean":"2023-10-07T23:16:28.007777+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:16:28.007777+0000","last_undegraded":"2023-10-07T23:16:28.007777+0000","last_fullsized":"2023-10-07T23:16:28.007777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008687+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:16:28.008687+0000","last_peered":"2023-10-07T23:16:28.008687+0000","last_clean":"2023-10-07T23:16:28.008687+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:16:28.008687+0000","last_undegraded":"2023-10-07T23:16:28.008687+0000","last_fullsized":"2023-10-07T23:16:28.008687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999386+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:16:27.999386+0000","last_peered":"2023-10-07T23:16:27.999386+0000","last_clean":"2023-10-07T23:16:27.999386+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:16:27.999386+0000","last_undegraded":"2023-10-07T23:16:27.999386+0000","last_fullsized":"2023-10-07T23:16:27.999386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007059+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:16:28.007059+0000","last_peered":"2023-10-07T23:16:28.007059+0000","last_clean":"2023-10-07T23:16:28.007059+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:16:28.007059+0000","last_undegraded":"2023-10-07T23:16:28.007059+0000","last_fullsized":"2023-10-07T23:16:28.007059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005716+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:16:28.005716+0000","last_peered":"2023-10-07T23:16:28.005716+0000","last_clean":"2023-10-07T23:16:28.005716+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:16:28.005716+0000","last_undegraded":"2023-10-07T23:16:28.005716+0000","last_fullsized":"2023-10-07T23:16:28.005716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999364+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:16:27.999364+0000","last_peered":"2023-10-07T23:16:27.999364+0000","last_clean":"2023-10-07T23:16:27.999364+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:16:27.999364+0000","last_undegraded":"2023-10-07T23:16:27.999364+0000","last_fullsized":"2023-10-07T23:16:27.999364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.015552+0000","last_change":"2023-10-07T23:16:29.015552+0000","last_active":"2023-10-07T23:16:29.015552+0000","last_peered":"2023-10-07T23:16:29.015552+0000","last_clean":"2023-10-07T23:16:29.015552+0000","last_became_active":"2023-10-07T23:16:29.010895+0000","last_became_peered":"2023-10-07T23:16:29.010895+0000","last_unstale":"2023-10-07T23:16:29.015552+0000","last_undegraded":"2023-10-07T23:16:29.015552+0000","last_fullsized":"2023-10-07T23:16:29.015552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:40.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.012361+0000","last_change":"2023-10-07T23:16:29.012361+0000","last_active":"2023-10-07T23:16:29.012361+0000","last_peered":"2023-10-07T23:16:29.012361+0000","last_clean":"2023-10-07T23:16:29.012361+0000","last_became_active":"2023-10-07T23:16:29.011918+0000","last_became_peered":"2023-10-07T23:16:29.011918+0000","last_unstale":"2023-10-07T23:16:29.012361+0000","last_undegraded":"2023-10-07T23:16:29.012361+0000","last_fullsized":"2023-10-07T23:16:29.012361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:32.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009934+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:16:28.009934+0000","last_peered":"2023-10-07T23:16:28.009934+0000","last_clean":"2023-10-07T23:16:28.009934+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:16:28.009934+0000","last_undegraded":"2023-10-07T23:16:28.009934+0000","last_fullsized":"2023-10-07T23:16:28.009934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001743+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:16:28.001743+0000","last_peered":"2023-10-07T23:16:28.001743+0000","last_clean":"2023-10-07T23:16:28.001743+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.001743+0000","last_undegraded":"2023-10-07T23:16:28.001743+0000","last_fullsized":"2023-10-07T23:16:28.001743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016218+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:16:28.016218+0000","last_peered":"2023-10-07T23:16:28.016218+0000","last_clean":"2023-10-07T23:16:28.016218+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:16:28.016218+0000","last_undegraded":"2023-10-07T23:16:28.016218+0000","last_fullsized":"2023-10-07T23:16:28.016218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.018993+0000","last_change":"2023-10-07T23:16:29.018993+0000","last_active":"2023-10-07T23:16:29.018993+0000","last_peered":"2023-10-07T23:16:29.018993+0000","last_clean":"2023-10-07T23:16:29.018993+0000","last_became_active":"2023-10-07T23:16:29.008350+0000","last_became_peered":"2023-10-07T23:16:29.008350+0000","last_unstale":"2023-10-07T23:16:29.018993+0000","last_undegraded":"2023-10-07T23:16:29.018993+0000","last_fullsized":"2023-10-07T23:16:29.018993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:16.236418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016364+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:16:28.016364+0000","last_peered":"2023-10-07T23:16:28.016364+0000","last_clean":"2023-10-07T23:16:28.016364+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:16:28.016364+0000","last_undegraded":"2023-10-07T23:16:28.016364+0000","last_fullsized":"2023-10-07T23:16:28.016364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006054+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:16:28.006054+0000","last_peered":"2023-10-07T23:16:28.006054+0000","last_clean":"2023-10-07T23:16:28.006054+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:16:28.006054+0000","last_undegraded":"2023-10-07T23:16:28.006054+0000","last_fullsized":"2023-10-07T23:16:28.006054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999879+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:16:27.999879+0000","last_peered":"2023-10-07T23:16:27.999879+0000","last_clean":"2023-10-07T23:16:27.999879+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:16:27.999879+0000","last_undegraded":"2023-10-07T23:16:27.999879+0000","last_fullsized":"2023-10-07T23:16:27.999879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012811+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:16:28.012811+0000","last_peered":"2023-10-07T23:16:28.012811+0000","last_clean":"2023-10-07T23:16:28.012811+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:16:28.012811+0000","last_undegraded":"2023-10-07T23:16:28.012811+0000","last_fullsized":"2023-10-07T23:16:28.012811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999561+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:16:27.999561+0000","last_peered":"2023-10-07T23:16:27.999561+0000","last_clean":"2023-10-07T23:16:27.999561+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:16:27.999561+0000","last_undegraded":"2023-10-07T23:16:27.999561+0000","last_fullsized":"2023-10-07T23:16:27.999561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005469+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:16:28.005469+0000","last_peered":"2023-10-07T23:16:28.005469+0000","last_clean":"2023-10-07T23:16:28.005469+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:16:28.005469+0000","last_undegraded":"2023-10-07T23:16:28.005469+0000","last_fullsized":"2023-10-07T23:16:28.005469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011194+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:16:28.011194+0000","last_peered":"2023-10-07T23:16:28.011194+0000","last_clean":"2023-10-07T23:16:28.011194+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:16:28.011194+0000","last_undegraded":"2023-10-07T23:16:28.011194+0000","last_fullsized":"2023-10-07T23:16:28.011194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006670+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:16:28.006670+0000","last_peered":"2023-10-07T23:16:28.006670+0000","last_clean":"2023-10-07T23:16:28.006670+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:16:28.006670+0000","last_undegraded":"2023-10-07T23:16:28.006670+0000","last_fullsized":"2023-10-07T23:16:28.006670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008316+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:16:28.008316+0000","last_peered":"2023-10-07T23:16:28.008316+0000","last_clean":"2023-10-07T23:16:28.008316+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:16:28.008316+0000","last_undegraded":"2023-10-07T23:16:28.008316+0000","last_fullsized":"2023-10-07T23:16:28.008316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002767+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:16:28.002767+0000","last_peered":"2023-10-07T23:16:28.002767+0000","last_clean":"2023-10-07T23:16:28.002767+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:16:28.002767+0000","last_undegraded":"2023-10-07T23:16:28.002767+0000","last_fullsized":"2023-10-07T23:16:28.002767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999422+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:16:27.999422+0000","last_peered":"2023-10-07T23:16:27.999422+0000","last_clean":"2023-10-07T23:16:27.999422+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:16:27.999422+0000","last_undegraded":"2023-10-07T23:16:27.999422+0000","last_fullsized":"2023-10-07T23:16:27.999422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006299+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:16:28.006299+0000","last_peered":"2023-10-07T23:16:28.006299+0000","last_clean":"2023-10-07T23:16:28.006299+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:16:28.006299+0000","last_undegraded":"2023-10-07T23:16:28.006299+0000","last_fullsized":"2023-10-07T23:16:28.006299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014270+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:16:28.014270+0000","last_peered":"2023-10-07T23:16:28.014270+0000","last_clean":"2023-10-07T23:16:28.014270+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:16:28.014270+0000","last_undegraded":"2023-10-07T23:16:28.014270+0000","last_fullsized":"2023-10-07T23:16:28.014270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999515+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:16:27.999515+0000","last_peered":"2023-10-07T23:16:27.999515+0000","last_clean":"2023-10-07T23:16:27.999515+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:16:27.999515+0000","last_undegraded":"2023-10-07T23:16:27.999515+0000","last_fullsized":"2023-10-07T23:16:27.999515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.049214+0000","last_change":"2023-10-07T23:16:29.049214+0000","last_active":"2023-10-07T23:16:29.049214+0000","last_peered":"2023-10-07T23:16:29.049214+0000","last_clean":"2023-10-07T23:16:29.049214+0000","last_became_active":"2023-10-07T23:16:29.044778+0000","last_became_peered":"2023-10-07T23:16:29.044778+0000","last_unstale":"2023-10-07T23:16:29.049214+0000","last_undegraded":"2023-10-07T23:16:29.049214+0000","last_fullsized":"2023-10-07T23:16:29.049214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:38.061993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007511+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:16:28.007511+0000","last_peered":"2023-10-07T23:16:28.007511+0000","last_clean":"2023-10-07T23:16:28.007511+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:16:28.007511+0000","last_undegraded":"2023-10-07T23:16:28.007511+0000","last_fullsized":"2023-10-07T23:16:28.007511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005590+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:16:28.005590+0000","last_peered":"2023-10-07T23:16:28.005590+0000","last_clean":"2023-10-07T23:16:28.005590+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:16:28.005590+0000","last_undegraded":"2023-10-07T23:16:28.005590+0000","last_fullsized":"2023-10-07T23:16:28.005590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:32.915735+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:16:32.915735+0000","last_peered":"2023-10-07T23:16:32.915735+0000","last_clean":"2023-10-07T23:16:32.915735+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:16:32.915735+0000","last_undegraded":"2023-10-07T23:16:32.915735+0000","last_fullsized":"2023-10-07T23:16:32.915735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007939+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:16:28.007939+0000","last_peered":"2023-10-07T23:16:28.007939+0000","last_clean":"2023-10-07T23:16:28.007939+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:16:28.007939+0000","last_undegraded":"2023-10-07T23:16:28.007939+0000","last_fullsized":"2023-10-07T23:16:28.007939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012043+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:16:28.012043+0000","last_peered":"2023-10-07T23:16:28.012043+0000","last_clean":"2023-10-07T23:16:28.012043+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:16:28.012043+0000","last_undegraded":"2023-10-07T23:16:28.012043+0000","last_fullsized":"2023-10-07T23:16:28.012043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005371+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:16:28.005371+0000","last_peered":"2023-10-07T23:16:28.005371+0000","last_clean":"2023-10-07T23:16:28.005371+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:16:28.005371+0000","last_undegraded":"2023-10-07T23:16:28.005371+0000","last_fullsized":"2023-10-07T23:16:28.005371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007323+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:16:28.007323+0000","last_peered":"2023-10-07T23:16:28.007323+0000","last_clean":"2023-10-07T23:16:28.007323+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:16:28.007323+0000","last_undegraded":"2023-10-07T23:16:28.007323+0000","last_fullsized":"2023-10-07T23:16:28.007323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003056+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:16:28.003056+0000","last_peered":"2023-10-07T23:16:28.003056+0000","last_clean":"2023-10-07T23:16:28.003056+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:16:28.003056+0000","last_undegraded":"2023-10-07T23:16:28.003056+0000","last_fullsized":"2023-10-07T23:16:28.003056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.051993+0000","last_change":"2023-10-07T23:16:29.051993+0000","last_active":"2023-10-07T23:16:29.051993+0000","last_peered":"2023-10-07T23:16:29.051993+0000","last_clean":"2023-10-07T23:16:29.051993+0000","last_became_active":"2023-10-07T23:16:29.046123+0000","last_became_peered":"2023-10-07T23:16:29.046123+0000","last_unstale":"2023-10-07T23:16:29.051993+0000","last_undegraded":"2023-10-07T23:16:29.051993+0000","last_fullsized":"2023-10-07T23:16:29.051993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:52.174959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008468+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:16:28.008468+0000","last_peered":"2023-10-07T23:16:28.008468+0000","last_clean":"2023-10-07T23:16:28.008468+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:16:28.008468+0000","last_undegraded":"2023-10-07T23:16:28.008468+0000","last_fullsized":"2023-10-07T23:16:28.008468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002713+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:16:28.002713+0000","last_peered":"2023-10-07T23:16:28.002713+0000","last_clean":"2023-10-07T23:16:28.002713+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:16:28.002713+0000","last_undegraded":"2023-10-07T23:16:28.002713+0000","last_fullsized":"2023-10-07T23:16:28.002713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006794+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:16:28.006794+0000","last_peered":"2023-10-07T23:16:28.006794+0000","last_clean":"2023-10-07T23:16:28.006794+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:16:28.006794+0000","last_undegraded":"2023-10-07T23:16:28.006794+0000","last_fullsized":"2023-10-07T23:16:28.006794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006214+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:16:28.006214+0000","last_peered":"2023-10-07T23:16:28.006214+0000","last_clean":"2023-10-07T23:16:28.006214+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:16:28.006214+0000","last_undegraded":"2023-10-07T23:16:28.006214+0000","last_fullsized":"2023-10-07T23:16:28.006214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012237+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:16:28.012237+0000","last_peered":"2023-10-07T23:16:28.012237+0000","last_clean":"2023-10-07T23:16:28.012237+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:16:28.012237+0000","last_undegraded":"2023-10-07T23:16:28.012237+0000","last_fullsized":"2023-10-07T23:16:28.012237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044932+0000","last_change":"2023-10-07T23:16:29.044932+0000","last_active":"2023-10-07T23:16:29.044932+0000","last_peered":"2023-10-07T23:16:29.044932+0000","last_clean":"2023-10-07T23:16:29.044932+0000","last_became_active":"2023-10-07T23:16:29.044454+0000","last_became_peered":"2023-10-07T23:16:29.044454+0000","last_unstale":"2023-10-07T23:16:29.044932+0000","last_undegraded":"2023-10-07T23:16:29.044932+0000","last_fullsized":"2023-10-07T23:16:29.044932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:55.975049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003230+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:16:28.003230+0000","last_peered":"2023-10-07T23:16:28.003230+0000","last_clean":"2023-10-07T23:16:28.003230+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:16:28.003230+0000","last_undegraded":"2023-10-07T23:16:28.003230+0000","last_fullsized":"2023-10-07T23:16:28.003230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008782+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:16:28.008782+0000","last_peered":"2023-10-07T23:16:28.008782+0000","last_clean":"2023-10-07T23:16:28.008782+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:16:28.008782+0000","last_undegraded":"2023-10-07T23:16:28.008782+0000","last_fullsized":"2023-10-07T23:16:28.008782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015307+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:16:28.015307+0000","last_peered":"2023-10-07T23:16:28.015307+0000","last_clean":"2023-10-07T23:16:28.015307+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:16:28.015307+0000","last_undegraded":"2023-10-07T23:16:28.015307+0000","last_fullsized":"2023-10-07T23:16:28.015307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003934+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:16:28.003934+0000","last_peered":"2023-10-07T23:16:28.003934+0000","last_clean":"2023-10-07T23:16:28.003934+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:16:28.003934+0000","last_undegraded":"2023-10-07T23:16:28.003934+0000","last_fullsized":"2023-10-07T23:16:28.003934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008400+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:16:28.008400+0000","last_peered":"2023-10-07T23:16:28.008400+0000","last_clean":"2023-10-07T23:16:28.008400+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:16:28.008400+0000","last_undegraded":"2023-10-07T23:16:28.008400+0000","last_fullsized":"2023-10-07T23:16:28.008400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005732+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:16:28.005732+0000","last_peered":"2023-10-07T23:16:28.005732+0000","last_clean":"2023-10-07T23:16:28.005732+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:16:28.005732+0000","last_undegraded":"2023-10-07T23:16:28.005732+0000","last_fullsized":"2023-10-07T23:16:28.005732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000267+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:16:28.000267+0000","last_peered":"2023-10-07T23:16:28.000267+0000","last_clean":"2023-10-07T23:16:28.000267+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:16:28.000267+0000","last_undegraded":"2023-10-07T23:16:28.000267+0000","last_fullsized":"2023-10-07T23:16:28.000267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003067+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:16:28.003067+0000","last_peered":"2023-10-07T23:16:28.003067+0000","last_clean":"2023-10-07T23:16:28.003067+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:16:28.003067+0000","last_undegraded":"2023-10-07T23:16:28.003067+0000","last_fullsized":"2023-10-07T23:16:28.003067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9805,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":418,"num_read_kb":227,"num_write":209,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":29415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383705,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":495361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.746},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71599999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0189999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.76000000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.82699999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.89900000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.94999999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298908,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446244,"statfs":{"total":95995035648,"available":95688953856,"internally_reserved":0,"allocated":1658880,"data_stored":1089665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45000000000000001,"15min":0.45200000000000001},"min":{"1min":0.31,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.83799999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.54200000000000004}]},{"osd":4,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72499999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.378}]},{"osd":5,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81699999999999995,"5min":1.159,"15min":1.159},"last":0.501},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0960000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.214,"15min":1.214},"last":0.624}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.62,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89600000000000002,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.57699999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98199999999999998,"5min":1.171,"15min":1.171},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.91400000000000003,"15min":0.94199999999999995},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.93700000000000006,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.68600000000000005}]},{"osd":11,"last update":"Sat Oct 7 23:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.64200000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68200000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.88,"5min":1.054,"15min":1.054},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.84699999999999998,"15min":0.86099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.325,"15min":1.325},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":1.278,"15min":1.278},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.67600000000000005,"15min":0.69799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76700000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":1.385,"15min":1.385},"last":0.61099999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.52800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97799999999999998,"5min":1.345,"15min":1.345},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.84799999999999998,"15min":0.873},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.82299999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.90100000000000002,"15min":0.92700000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97599999999999998,"5min":1.579,"15min":1.579},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.96399999999999997,"15min":0.99299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0649999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.113,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.018,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2310000000000001,"5min":1.603,"15min":1.603},"last":0.501}]},{"osd":8,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.026,"15min":1.056},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.038,"5min":1.782,"15min":1.782},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.96599999999999997,"15min":0.998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0529999999999999,"5min":1.956,"15min":1.956},"last":0.625}]},{"osd":10,"last update":"Sat Oct 7 23:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.008,"15min":1.0249999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2849999999999999,"5min":1.647,"15min":1.647},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.0009999999999999,"15min":1.0189999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.165,"5min":1.663,"15min":1.663},"last":0.59299999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.747},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68500000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":1044480,"data_stored":495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39700000000000002}]},{"osd":3,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75}]},{"osd":4,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.496}]},{"osd":5,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.434},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.59299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.022,"5min":1.597,"15min":1.597},"last":0.72199999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1,"15min":1},"last":0.55400000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.59899999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92900000000000005,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96399999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.72699999999999998,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.61899999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298888,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446264,"statfs":{"total":95995035648,"available":95688974336,"internally_reserved":0,"allocated":1638400,"data_stored":1087555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57999999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81899999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":1.51,"15min":1.51},"last":0.48099999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82599999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.435},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41099999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63200000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.38}]},{"osd":5,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.496},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84699999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.48299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.47099999999999997}]},{"osd":8,"last update":"Sat Oct 7 23:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.92800000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.50600000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78100000000000003,"15min":0.79100000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.03,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.39700000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98399999999999999,"5min":1.056,"15min":1.056},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0029999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.60799999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57199999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":494596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70999999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70999999999999996,"5min":0.878,"15min":0.878},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.70999999999999996,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.5700000000000003,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.56299999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.879}]},{"osd":3,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.877,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.95599999999999996,"5min":1.014,"15min":1.014},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75,"15min":0.751},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.746}]},{"osd":5,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.099,"15min":1.099},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79700000000000004,"15min":0.77900000000000003},"min":{"1min":0.53100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.875,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0700000000000001,"5min":6.524,"15min":6.524},"last":0.76500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85799999999999998,"15min":0.85599999999999998},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.0509999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88800000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0580000000000001,"5min":1.165,"15min":1.165},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.1619999999999999,"15min":1.206},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1679999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.113,"15min":1.1459999999999999},"min":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.472,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72299999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.378}]},{"osd":2,"last update":"Sat Oct 7 23:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.56899999999999995,"15min":0.52600000000000002},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60599999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55900000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.42599999999999999},"min":{"1min":0.28499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.437}]},{"osd":5,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.60799999999999998,"15min":0.54800000000000004},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57999999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.39200000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41799999999999998}]},{"osd":8,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.628},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48499999999999999}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.621},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.54600000000000004}]},{"osd":10,"last update":"Sat Oct 7 23:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.751,"15min":0.754},"min":{"1min":0.56999999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.65300000000000002}]},{"osd":11,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.67500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":6.5650000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":1.022,"15min":1.022},"last":5.4249999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.022,"5min":1.212,"15min":1.212},"last":6.5990000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.48199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.879}]},{"osd":3,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":5.5049999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.92500000000000004}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":1.105},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":5.4669999999999996}]},{"osd":5,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.52100000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":1.155,"15min":1.155},"last":5.3769999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":5.5789999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.1830000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.2130000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2150000000000001,"5min":1.323,"15min":1.323},"last":5.6180000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":5.0609999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88800000000000001,"15min":0.88900000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":5.5439999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":6.6539999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446836,"statfs":{"total":95995035648,"available":95689560064,"internally_reserved":0,"allocated":1052672,"data_stored":499485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54500000000000004,"15min":0.499},"min":{"1min":0.436,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84999999999999998,"5min":0.871,"15min":0.871},"last":0.61299999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.47299999999999998,"15min":0.44500000000000001},"min":{"1min":0.38300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.74199999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.36299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.40600000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67500000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72799999999999998}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.628,"15min":0.60099999999999998},"min":{"1min":0.59099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.55800000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.65100000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.72999999999999998,"15min":0.70499999999999996},"min":{"1min":0.57699999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.747,"15min":0.72199999999999998},"min":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.79000000000000004}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76800000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":1040384,"data_stored":494675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.46999999999999997,"15min":0.41599999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.51800000000000002,"15min":0.45200000000000001},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.68000000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.60299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89500000000000002,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.625}]},{"osd":2,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.69599999999999995,"15min":0.625},"min":{"1min":0.46800000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69099999999999995,"15min":0.622},"min":{"1min":0.56100000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.95099999999999996,"5min":1.218,"15min":1.218},"last":0.501}]},{"osd":4,"last update":"Sat Oct 7 23:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.71399999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.71699999999999997,"15min":0.67700000000000005},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.0069999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70899999999999996,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1459999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.374,"15min":0.374},"max":{"1min":1.048,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":1.1319999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.72099999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.089,"5min":1.335,"15min":1.335},"last":0.73699999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.71399999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2230000000000001,"5min":1.288,"15min":1.288},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0029999999999999,"15min":1.0189999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.169,"5min":1.51,"15min":1.51},"last":0.88}]},{"osd":10,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0920000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83399999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.84099999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":494897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.36499999999999999},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.38800000000000001,"15min":0.35299999999999998},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.27700000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.49399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.52900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.46999999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.39400000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.55300000000000005},"min":{"1min":0.52500000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60599999999999998,"15min":0.55400000000000005},"min":{"1min":0.58399999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.55500000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.63700000000000001},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.38100000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.55700000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.61599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.36299999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54300000000000004}]},{"osd":9,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.80200000000000005,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.496},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47899999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.752,"15min":0.749},"min":{"1min":0.66100000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81799999999999995},"min":{"1min":0.73699999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.435}]},{"osd":11,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.66800000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446256,"statfs":{"total":95995035648,"available":95688966144,"internally_reserved":0,"allocated":1646592,"data_stored":1086978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.47999999999999998},"min":{"1min":0.30299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.48399999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":3.6230000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53700000000000003,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46600000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.39300000000000002,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40699999999999997,"15min":0.378},"min":{"1min":0.27000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.35999999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.60799999999999998,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.60299999999999998,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Oct 7 23:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.63}]},{"osd":7,"last update":"Sat Oct 7 23:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55200000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.47899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.499}]},{"osd":9,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1,"5min":1,"15min":1},"last":0.67900000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93899999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:16:37.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-07T23:16:38.256 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:38.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:38 smithi123 ceph-mon[122926]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:16:38.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:16:38.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:16:38.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:38 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:38 smithi086 ceph-mon[122131]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:16:38.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:16:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:16:38.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:38 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:38.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:38 smithi067 ceph-mon[82723]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:16:38.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-07T23:16:38.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-07T23:16:38.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:38 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' 2023-10-07T23:16:39.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:39 smithi067 ceph-mon[82723]: pgmap v326: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:39.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:39 smithi123 ceph-mon[122926]: pgmap v326: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:39.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:39 smithi086 ceph-mon[122131]: pgmap v326: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 -- 172.21.15.67:0/3029053567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4070520 msgr2=0x7f50a4070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:39.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/3029053567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4070520 0x7f50a4070990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50940049c0 tx=0x7f5094009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 -- 172.21.15.67:0/3029053567 shutdown_connections 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/3029053567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a410fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/3029053567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a4071c10 0x7f50a406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/3029053567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4070520 0x7f50a4070990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.674+0000 7f50ab6ab700 1 -- 172.21.15.67:0/3029053567 >> 172.21.15.67:0/3029053567 conn(0x7f50a40f89f0 msgr2=0x7f50a40fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.675+0000 7f50ab6ab700 1 -- 172.21.15.67:0/3029053567 shutdown_connections 2023-10-07T23:16:39.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.675+0000 7f50ab6ab700 1 -- 172.21.15.67:0/3029053567 wait complete. 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.675+0000 7f50ab6ab700 1 Processor -- start 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.675+0000 7f50ab6ab700 1 -- start start 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50ab6ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a4070520 0x7f50a411a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50ab6ab700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4071c10 0x7f50a411a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50ab6ab700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a4114180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50ab6ab700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41147a0 con 0x7f50a410d9b0 2023-10-07T23:16:39.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50ab6ab700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4114910 con 0x7f50a4071c10 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50ab6ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4114ad0 con 0x7f50a4070520 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50a9c48700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a4114180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50a9c48700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a4114180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48702/0 (socket says 172.21.15.67:48702) 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50a9c48700 1 -- 172.21.15.67:0/2155273469 learned_addr learned my addr 172.21.15.67:0/2155273469 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50a8c46700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4071c10 0x7f50a411a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.676+0000 7f50a9447700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a4070520 0x7f50a411a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a8c46700 1 -- 172.21.15.67:0/2155273469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a4070520 msgr2=0x7f50a411a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a8c46700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a4070520 0x7f50a411a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a8c46700 1 -- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 msgr2=0x7f50a4114180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a8c46700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a4114180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a8c46700 1 -- 172.21.15.67:0/2155273469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094005040 con 0x7f50a4071c10 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a9c48700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a4114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:39.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50a8c46700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4071c10 0x7f50a411a610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50940049c0 tx=0x7f5094009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094020070 con 0x7f50a4071c10 2023-10-07T23:16:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5094009770 con 0x7f50a4071c10 2023-10-07T23:16:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400e040 con 0x7f50a4071c10 2023-10-07T23:16:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a4114d50 con 0x7f50a4071c10 2023-10-07T23:16:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.677+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a4120f00 con 0x7f50a4071c10 2023-10-07T23:16:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.678+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a410c780 con 0x7f50a4071c10 2023-10-07T23:16:39.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.680+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f50940092a0 con 0x7f50a4071c10 2023-10-07T23:16:39.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.682+0000 7f5099ffb700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509009b770 0x7f509009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:39.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.682+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f50940c6610 con 0x7f50a4071c10 2023-10-07T23:16:39.683 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.682+0000 7f50a9447700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509009b770 0x7f509009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:39.683 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.683+0000 7f50a9447700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509009b770 0x7f509009dc20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009b80 tx=0x7f50a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:39.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.684+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f5094091ac0 con 0x7f50a4071c10 2023-10-07T23:16:39.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.885+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50a4115720 con 0x7f509009b770 2023-10-07T23:16:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.898+0000 7f5099ffb700 1 -- 172.21.15.67:0/2155273469 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426829 (secure 0 0 0) 0x7f50a4115720 con 0x7f509009b770 2023-10-07T23:16:39.905 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509009b770 msgr2=0x7f509009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509009b770 0x7f509009dc20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009b80 tx=0x7f50a000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4071c10 msgr2=0x7f50a411a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4071c10 0x7f50a411a610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50940049c0 tx=0x7f5094009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 shutdown_connections 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f509009b770 0x7f509009dc20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50a410d9b0 0x7f50a4114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50a4070520 0x7f50a411a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 --2- 172.21.15.67:0/2155273469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50a4071c10 0x7f50a411a610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 >> 172.21.15.67:0/2155273469 conn(0x7f50a40f89f0 msgr2=0x7f50a4109900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 shutdown_connections 2023-10-07T23:16:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:39.904+0000 7f50ab6ab700 1 -- 172.21.15.67:0/2155273469 wait complete. 2023-10-07T23:16:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:16:40.339 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-10-07T23:16:37.939032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600173,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":579,"num_read_kb":351,"num_write":390,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3581412,"kb_used_data":13956,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121360412,"statfs":{"total":1151940427776,"available":1148273061888,"internally_reserved":0,"allocated":14290944,"data_stored":7729521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.940661"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004139+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:16:28.004139+0000","last_peered":"2023-10-07T23:16:28.004139+0000","last_clean":"2023-10-07T23:16:28.004139+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:16:28.004139+0000","last_undegraded":"2023-10-07T23:16:28.004139+0000","last_fullsized":"2023-10-07T23:16:28.004139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012736+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:16:28.012736+0000","last_peered":"2023-10-07T23:16:28.012736+0000","last_clean":"2023-10-07T23:16:28.012736+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:16:28.012736+0000","last_undegraded":"2023-10-07T23:16:28.012736+0000","last_fullsized":"2023-10-07T23:16:28.012736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012507+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:16:28.012507+0000","last_peered":"2023-10-07T23:16:28.012507+0000","last_clean":"2023-10-07T23:16:28.012507+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:16:28.012507+0000","last_undegraded":"2023-10-07T23:16:28.012507+0000","last_fullsized":"2023-10-07T23:16:28.012507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999764+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:16:27.999764+0000","last_peered":"2023-10-07T23:16:27.999764+0000","last_clean":"2023-10-07T23:16:27.999764+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:16:27.999764+0000","last_undegraded":"2023-10-07T23:16:27.999764+0000","last_fullsized":"2023-10-07T23:16:27.999764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011235+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:16:28.011235+0000","last_peered":"2023-10-07T23:16:28.011235+0000","last_clean":"2023-10-07T23:16:28.011235+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:16:28.011235+0000","last_undegraded":"2023-10-07T23:16:28.011235+0000","last_fullsized":"2023-10-07T23:16:28.011235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003946+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:16:28.003946+0000","last_peered":"2023-10-07T23:16:28.003946+0000","last_clean":"2023-10-07T23:16:28.003946+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:16:28.003946+0000","last_undegraded":"2023-10-07T23:16:28.003946+0000","last_fullsized":"2023-10-07T23:16:28.003946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007260+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:16:28.007260+0000","last_peered":"2023-10-07T23:16:28.007260+0000","last_clean":"2023-10-07T23:16:28.007260+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:16:28.007260+0000","last_undegraded":"2023-10-07T23:16:28.007260+0000","last_fullsized":"2023-10-07T23:16:28.007260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015359+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:16:28.015359+0000","last_peered":"2023-10-07T23:16:28.015359+0000","last_clean":"2023-10-07T23:16:28.015359+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:16:28.015359+0000","last_undegraded":"2023-10-07T23:16:28.015359+0000","last_fullsized":"2023-10-07T23:16:28.015359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006337+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:16:28.006337+0000","last_peered":"2023-10-07T23:16:28.006337+0000","last_clean":"2023-10-07T23:16:28.006337+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:16:28.006337+0000","last_undegraded":"2023-10-07T23:16:28.006337+0000","last_fullsized":"2023-10-07T23:16:28.006337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003351+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:16:28.003351+0000","last_peered":"2023-10-07T23:16:28.003351+0000","last_clean":"2023-10-07T23:16:28.003351+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:16:28.003351+0000","last_undegraded":"2023-10-07T23:16:28.003351+0000","last_fullsized":"2023-10-07T23:16:28.003351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007883+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:16:28.007883+0000","last_peered":"2023-10-07T23:16:28.007883+0000","last_clean":"2023-10-07T23:16:28.007883+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:16:28.007883+0000","last_undegraded":"2023-10-07T23:16:28.007883+0000","last_fullsized":"2023-10-07T23:16:28.007883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999630+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:16:27.999630+0000","last_peered":"2023-10-07T23:16:27.999630+0000","last_clean":"2023-10-07T23:16:27.999630+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:16:27.999630+0000","last_undegraded":"2023-10-07T23:16:27.999630+0000","last_fullsized":"2023-10-07T23:16:27.999630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000842+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:16:28.000842+0000","last_peered":"2023-10-07T23:16:28.000842+0000","last_clean":"2023-10-07T23:16:28.000842+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:16:28.000842+0000","last_undegraded":"2023-10-07T23:16:28.000842+0000","last_fullsized":"2023-10-07T23:16:28.000842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017901+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:16:28.017901+0000","last_peered":"2023-10-07T23:16:28.017901+0000","last_clean":"2023-10-07T23:16:28.017901+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:16:28.017901+0000","last_undegraded":"2023-10-07T23:16:28.017901+0000","last_fullsized":"2023-10-07T23:16:28.017901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011942+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:16:28.011942+0000","last_peered":"2023-10-07T23:16:28.011942+0000","last_clean":"2023-10-07T23:16:28.011942+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:16:28.011942+0000","last_undegraded":"2023-10-07T23:16:28.011942+0000","last_fullsized":"2023-10-07T23:16:28.011942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005454+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:16:28.005454+0000","last_peered":"2023-10-07T23:16:28.005454+0000","last_clean":"2023-10-07T23:16:28.005454+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:16:28.005454+0000","last_undegraded":"2023-10-07T23:16:28.005454+0000","last_fullsized":"2023-10-07T23:16:28.005454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004354+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:16:28.004354+0000","last_peered":"2023-10-07T23:16:28.004354+0000","last_clean":"2023-10-07T23:16:28.004354+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:16:28.004354+0000","last_undegraded":"2023-10-07T23:16:28.004354+0000","last_fullsized":"2023-10-07T23:16:28.004354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009520+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:16:28.009520+0000","last_peered":"2023-10-07T23:16:28.009520+0000","last_clean":"2023-10-07T23:16:28.009520+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.009520+0000","last_undegraded":"2023-10-07T23:16:28.009520+0000","last_fullsized":"2023-10-07T23:16:28.009520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008489+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:16:28.008489+0000","last_peered":"2023-10-07T23:16:28.008489+0000","last_clean":"2023-10-07T23:16:28.008489+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:16:28.008489+0000","last_undegraded":"2023-10-07T23:16:28.008489+0000","last_fullsized":"2023-10-07T23:16:28.008489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006630+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:16:28.006630+0000","last_peered":"2023-10-07T23:16:28.006630+0000","last_clean":"2023-10-07T23:16:28.006630+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:16:28.006630+0000","last_undegraded":"2023-10-07T23:16:28.006630+0000","last_fullsized":"2023-10-07T23:16:28.006630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:31.914321+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:16:31.914321+0000","last_peered":"2023-10-07T23:16:31.914321+0000","last_clean":"2023-10-07T23:16:31.914321+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:16:31.914321+0000","last_undegraded":"2023-10-07T23:16:31.914321+0000","last_fullsized":"2023-10-07T23:16:31.914321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014500+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:16:28.014500+0000","last_peered":"2023-10-07T23:16:28.014500+0000","last_clean":"2023-10-07T23:16:28.014500+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:16:28.014500+0000","last_undegraded":"2023-10-07T23:16:28.014500+0000","last_fullsized":"2023-10-07T23:16:28.014500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999223+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:16:27.999223+0000","last_peered":"2023-10-07T23:16:27.999223+0000","last_clean":"2023-10-07T23:16:27.999223+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:16:27.999223+0000","last_undegraded":"2023-10-07T23:16:27.999223+0000","last_fullsized":"2023-10-07T23:16:27.999223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014597+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:16:28.014597+0000","last_peered":"2023-10-07T23:16:28.014597+0000","last_clean":"2023-10-07T23:16:28.014597+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:16:28.014597+0000","last_undegraded":"2023-10-07T23:16:28.014597+0000","last_fullsized":"2023-10-07T23:16:28.014597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012905+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:16:28.012905+0000","last_peered":"2023-10-07T23:16:28.012905+0000","last_clean":"2023-10-07T23:16:28.012905+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:16:28.012905+0000","last_undegraded":"2023-10-07T23:16:28.012905+0000","last_fullsized":"2023-10-07T23:16:28.012905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999373+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:16:27.999373+0000","last_peered":"2023-10-07T23:16:27.999373+0000","last_clean":"2023-10-07T23:16:27.999373+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:16:27.999373+0000","last_undegraded":"2023-10-07T23:16:27.999373+0000","last_fullsized":"2023-10-07T23:16:27.999373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010037+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:16:28.010037+0000","last_peered":"2023-10-07T23:16:28.010037+0000","last_clean":"2023-10-07T23:16:28.010037+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:16:28.010037+0000","last_undegraded":"2023-10-07T23:16:28.010037+0000","last_fullsized":"2023-10-07T23:16:28.010037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005917+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:16:28.005917+0000","last_peered":"2023-10-07T23:16:28.005917+0000","last_clean":"2023-10-07T23:16:28.005917+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:16:28.005917+0000","last_undegraded":"2023-10-07T23:16:28.005917+0000","last_fullsized":"2023-10-07T23:16:28.005917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999220+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:16:27.999220+0000","last_peered":"2023-10-07T23:16:27.999220+0000","last_clean":"2023-10-07T23:16:27.999220+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:16:27.999220+0000","last_undegraded":"2023-10-07T23:16:27.999220+0000","last_fullsized":"2023-10-07T23:16:27.999220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011048+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:16:28.011048+0000","last_peered":"2023-10-07T23:16:28.011048+0000","last_clean":"2023-10-07T23:16:28.011048+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:16:28.011048+0000","last_undegraded":"2023-10-07T23:16:28.011048+0000","last_fullsized":"2023-10-07T23:16:28.011048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005339+0000","last_change":"2023-10-07T23:15:27.197818+0000","last_active":"2023-10-07T23:16:28.005339+0000","last_peered":"2023-10-07T23:16:28.005339+0000","last_clean":"2023-10-07T23:16:28.005339+0000","last_became_active":"2023-10-07T23:15:27.197522+0000","last_became_peered":"2023-10-07T23:15:27.197522+0000","last_unstale":"2023-10-07T23:16:28.005339+0000","last_undegraded":"2023-10-07T23:16:28.005339+0000","last_fullsized":"2023-10-07T23:16:28.005339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:11.128855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008264+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:16:28.008264+0000","last_peered":"2023-10-07T23:16:28.008264+0000","last_clean":"2023-10-07T23:16:28.008264+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:16:28.008264+0000","last_undegraded":"2023-10-07T23:16:28.008264+0000","last_fullsized":"2023-10-07T23:16:28.008264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:35.901608+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:16:35.901608+0000","last_peered":"2023-10-07T23:16:35.901608+0000","last_clean":"2023-10-07T23:16:35.901608+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:16:35.901608+0000","last_undegraded":"2023-10-07T23:16:35.901608+0000","last_fullsized":"2023-10-07T23:16:35.901608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010625+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:16:28.010625+0000","last_peered":"2023-10-07T23:16:28.010625+0000","last_clean":"2023-10-07T23:16:28.010625+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:16:28.010625+0000","last_undegraded":"2023-10-07T23:16:28.010625+0000","last_fullsized":"2023-10-07T23:16:28.010625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.013091+0000","last_change":"2023-10-07T23:16:29.013091+0000","last_active":"2023-10-07T23:16:29.013091+0000","last_peered":"2023-10-07T23:16:29.013091+0000","last_clean":"2023-10-07T23:16:29.013091+0000","last_became_active":"2023-10-07T23:16:29.012680+0000","last_became_peered":"2023-10-07T23:16:29.012680+0000","last_unstale":"2023-10-07T23:16:29.013091+0000","last_undegraded":"2023-10-07T23:16:29.013091+0000","last_fullsized":"2023-10-07T23:16:29.013091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:13.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.050161+0000","last_change":"2023-10-07T23:16:29.050161+0000","last_active":"2023-10-07T23:16:29.050161+0000","last_peered":"2023-10-07T23:16:29.050161+0000","last_clean":"2023-10-07T23:16:29.050161+0000","last_became_active":"2023-10-07T23:16:29.044559+0000","last_became_peered":"2023-10-07T23:16:29.044559+0000","last_unstale":"2023-10-07T23:16:29.050161+0000","last_undegraded":"2023-10-07T23:16:29.050161+0000","last_fullsized":"2023-10-07T23:16:29.050161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:18.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006444+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:16:28.006444+0000","last_peered":"2023-10-07T23:16:28.006444+0000","last_clean":"2023-10-07T23:16:28.006444+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:16:28.006444+0000","last_undegraded":"2023-10-07T23:16:28.006444+0000","last_fullsized":"2023-10-07T23:16:28.006444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045657+0000","last_change":"2023-10-07T23:16:29.045657+0000","last_active":"2023-10-07T23:16:29.045657+0000","last_peered":"2023-10-07T23:16:29.045657+0000","last_clean":"2023-10-07T23:16:29.045657+0000","last_became_active":"2023-10-07T23:16:29.045268+0000","last_became_peered":"2023-10-07T23:16:29.045268+0000","last_unstale":"2023-10-07T23:16:29.045657+0000","last_undegraded":"2023-10-07T23:16:29.045657+0000","last_fullsized":"2023-10-07T23:16:29.045657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:34.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999559+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:16:27.999559+0000","last_peered":"2023-10-07T23:16:27.999559+0000","last_clean":"2023-10-07T23:16:27.999559+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:16:27.999559+0000","last_undegraded":"2023-10-07T23:16:27.999559+0000","last_fullsized":"2023-10-07T23:16:27.999559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016282+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:16:28.016282+0000","last_peered":"2023-10-07T23:16:28.016282+0000","last_clean":"2023-10-07T23:16:28.016282+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:16:28.016282+0000","last_undegraded":"2023-10-07T23:16:28.016282+0000","last_fullsized":"2023-10-07T23:16:28.016282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012663+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:16:28.012663+0000","last_peered":"2023-10-07T23:16:28.012663+0000","last_clean":"2023-10-07T23:16:28.012663+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:16:28.012663+0000","last_undegraded":"2023-10-07T23:16:28.012663+0000","last_fullsized":"2023-10-07T23:16:28.012663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003839+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:16:28.003839+0000","last_peered":"2023-10-07T23:16:28.003839+0000","last_clean":"2023-10-07T23:16:28.003839+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:16:28.003839+0000","last_undegraded":"2023-10-07T23:16:28.003839+0000","last_fullsized":"2023-10-07T23:16:28.003839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005155+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:16:28.005155+0000","last_peered":"2023-10-07T23:16:28.005155+0000","last_clean":"2023-10-07T23:16:28.005155+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:16:28.005155+0000","last_undegraded":"2023-10-07T23:16:28.005155+0000","last_fullsized":"2023-10-07T23:16:28.005155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008218+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:16:28.008218+0000","last_peered":"2023-10-07T23:16:28.008218+0000","last_clean":"2023-10-07T23:16:28.008218+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:16:28.008218+0000","last_undegraded":"2023-10-07T23:16:28.008218+0000","last_fullsized":"2023-10-07T23:16:28.008218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:36.902357+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:16:36.902357+0000","last_peered":"2023-10-07T23:16:36.902357+0000","last_clean":"2023-10-07T23:16:36.902357+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:16:36.902357+0000","last_undegraded":"2023-10-07T23:16:36.902357+0000","last_fullsized":"2023-10-07T23:16:36.902357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006164+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:16:28.006164+0000","last_peered":"2023-10-07T23:16:28.006164+0000","last_clean":"2023-10-07T23:16:28.006164+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:16:28.006164+0000","last_undegraded":"2023-10-07T23:16:28.006164+0000","last_fullsized":"2023-10-07T23:16:28.006164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004871+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:16:28.004871+0000","last_peered":"2023-10-07T23:16:28.004871+0000","last_clean":"2023-10-07T23:16:28.004871+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:16:28.004871+0000","last_undegraded":"2023-10-07T23:16:28.004871+0000","last_fullsized":"2023-10-07T23:16:28.004871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007922+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:16:28.007922+0000","last_peered":"2023-10-07T23:16:28.007922+0000","last_clean":"2023-10-07T23:16:28.007922+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:16:28.007922+0000","last_undegraded":"2023-10-07T23:16:28.007922+0000","last_fullsized":"2023-10-07T23:16:28.007922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017142+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:16:28.017142+0000","last_peered":"2023-10-07T23:16:28.017142+0000","last_clean":"2023-10-07T23:16:28.017142+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:16:28.017142+0000","last_undegraded":"2023-10-07T23:16:28.017142+0000","last_fullsized":"2023-10-07T23:16:28.017142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999614+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:16:27.999614+0000","last_peered":"2023-10-07T23:16:27.999614+0000","last_clean":"2023-10-07T23:16:27.999614+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:16:27.999614+0000","last_undegraded":"2023-10-07T23:16:27.999614+0000","last_fullsized":"2023-10-07T23:16:27.999614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005267+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:16:28.005267+0000","last_peered":"2023-10-07T23:16:28.005267+0000","last_clean":"2023-10-07T23:16:28.005267+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:16:28.005267+0000","last_undegraded":"2023-10-07T23:16:28.005267+0000","last_fullsized":"2023-10-07T23:16:28.005267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006062+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:16:28.006062+0000","last_peered":"2023-10-07T23:16:28.006062+0000","last_clean":"2023-10-07T23:16:28.006062+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:16:28.006062+0000","last_undegraded":"2023-10-07T23:16:28.006062+0000","last_fullsized":"2023-10-07T23:16:28.006062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005402+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:16:28.005402+0000","last_peered":"2023-10-07T23:16:28.005402+0000","last_clean":"2023-10-07T23:16:28.005402+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:16:28.005402+0000","last_undegraded":"2023-10-07T23:16:28.005402+0000","last_fullsized":"2023-10-07T23:16:28.005402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'14","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:36.902436+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:16:36.902436+0000","last_peered":"2023-10-07T23:16:36.902436+0000","last_clean":"2023-10-07T23:16:36.902436+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:16:36.902436+0000","last_undegraded":"2023-10-07T23:16:36.902436+0000","last_fullsized":"2023-10-07T23:16:36.902436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":48,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009919+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:16:28.009919+0000","last_peered":"2023-10-07T23:16:28.009919+0000","last_clean":"2023-10-07T23:16:28.009919+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:16:28.009919+0000","last_undegraded":"2023-10-07T23:16:28.009919+0000","last_fullsized":"2023-10-07T23:16:28.009919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005504+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:16:28.005504+0000","last_peered":"2023-10-07T23:16:28.005504+0000","last_clean":"2023-10-07T23:16:28.005504+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:16:28.005504+0000","last_undegraded":"2023-10-07T23:16:28.005504+0000","last_fullsized":"2023-10-07T23:16:28.005504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007745+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:16:28.007745+0000","last_peered":"2023-10-07T23:16:28.007745+0000","last_clean":"2023-10-07T23:16:28.007745+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:16:28.007745+0000","last_undegraded":"2023-10-07T23:16:28.007745+0000","last_fullsized":"2023-10-07T23:16:28.007745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999913+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:16:27.999913+0000","last_peered":"2023-10-07T23:16:27.999913+0000","last_clean":"2023-10-07T23:16:27.999913+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:16:27.999913+0000","last_undegraded":"2023-10-07T23:16:27.999913+0000","last_fullsized":"2023-10-07T23:16:27.999913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002941+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:16:28.002941+0000","last_peered":"2023-10-07T23:16:28.002941+0000","last_clean":"2023-10-07T23:16:28.002941+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:16:28.002941+0000","last_undegraded":"2023-10-07T23:16:28.002941+0000","last_fullsized":"2023-10-07T23:16:28.002941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005850+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:16:28.005850+0000","last_peered":"2023-10-07T23:16:28.005850+0000","last_clean":"2023-10-07T23:16:28.005850+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:16:28.005850+0000","last_undegraded":"2023-10-07T23:16:28.005850+0000","last_fullsized":"2023-10-07T23:16:28.005850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008325+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:16:28.008325+0000","last_peered":"2023-10-07T23:16:28.008325+0000","last_clean":"2023-10-07T23:16:28.008325+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:16:28.008325+0000","last_undegraded":"2023-10-07T23:16:28.008325+0000","last_fullsized":"2023-10-07T23:16:28.008325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044573+0000","last_change":"2023-10-07T23:16:29.044573+0000","last_active":"2023-10-07T23:16:29.044573+0000","last_peered":"2023-10-07T23:16:29.044573+0000","last_clean":"2023-10-07T23:16:29.044573+0000","last_became_active":"2023-10-07T23:16:29.044228+0000","last_became_peered":"2023-10-07T23:16:29.044228+0000","last_unstale":"2023-10-07T23:16:29.044573+0000","last_undegraded":"2023-10-07T23:16:29.044573+0000","last_fullsized":"2023-10-07T23:16:29.044573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:31.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:35.901016+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:16:35.901016+0000","last_peered":"2023-10-07T23:16:35.901016+0000","last_clean":"2023-10-07T23:16:35.901016+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:16:35.901016+0000","last_undegraded":"2023-10-07T23:16:35.901016+0000","last_fullsized":"2023-10-07T23:16:35.901016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007188+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:16:28.007188+0000","last_peered":"2023-10-07T23:16:28.007188+0000","last_clean":"2023-10-07T23:16:28.007188+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:16:28.007188+0000","last_undegraded":"2023-10-07T23:16:28.007188+0000","last_fullsized":"2023-10-07T23:16:28.007188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003129+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:16:28.003129+0000","last_peered":"2023-10-07T23:16:28.003129+0000","last_clean":"2023-10-07T23:16:28.003129+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:16:28.003129+0000","last_undegraded":"2023-10-07T23:16:28.003129+0000","last_fullsized":"2023-10-07T23:16:28.003129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012034+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:16:28.012034+0000","last_peered":"2023-10-07T23:16:28.012034+0000","last_clean":"2023-10-07T23:16:28.012034+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:16:28.012034+0000","last_undegraded":"2023-10-07T23:16:28.012034+0000","last_fullsized":"2023-10-07T23:16:28.012034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003744+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:16:28.003744+0000","last_peered":"2023-10-07T23:16:28.003744+0000","last_clean":"2023-10-07T23:16:28.003744+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:16:28.003744+0000","last_undegraded":"2023-10-07T23:16:28.003744+0000","last_fullsized":"2023-10-07T23:16:28.003744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003691+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:16:28.003691+0000","last_peered":"2023-10-07T23:16:28.003691+0000","last_clean":"2023-10-07T23:16:28.003691+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:16:28.003691+0000","last_undegraded":"2023-10-07T23:16:28.003691+0000","last_fullsized":"2023-10-07T23:16:28.003691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012511+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:16:28.012511+0000","last_peered":"2023-10-07T23:16:28.012511+0000","last_clean":"2023-10-07T23:16:28.012511+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:16:28.012511+0000","last_undegraded":"2023-10-07T23:16:28.012511+0000","last_fullsized":"2023-10-07T23:16:28.012511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003423+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:16:28.003423+0000","last_peered":"2023-10-07T23:16:28.003423+0000","last_clean":"2023-10-07T23:16:28.003423+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:16:28.003423+0000","last_undegraded":"2023-10-07T23:16:28.003423+0000","last_fullsized":"2023-10-07T23:16:28.003423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010581+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:16:28.010581+0000","last_peered":"2023-10-07T23:16:28.010581+0000","last_clean":"2023-10-07T23:16:28.010581+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:16:28.010581+0000","last_undegraded":"2023-10-07T23:16:28.010581+0000","last_fullsized":"2023-10-07T23:16:28.010581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006472+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:16:28.006472+0000","last_peered":"2023-10-07T23:16:28.006472+0000","last_clean":"2023-10-07T23:16:28.006472+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:16:28.006472+0000","last_undegraded":"2023-10-07T23:16:28.006472+0000","last_fullsized":"2023-10-07T23:16:28.006472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003225+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:16:28.003225+0000","last_peered":"2023-10-07T23:16:28.003225+0000","last_clean":"2023-10-07T23:16:28.003225+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:16:28.003225+0000","last_undegraded":"2023-10-07T23:16:28.003225+0000","last_fullsized":"2023-10-07T23:16:28.003225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001591+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:16:28.001591+0000","last_peered":"2023-10-07T23:16:28.001591+0000","last_clean":"2023-10-07T23:16:28.001591+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:16:28.001591+0000","last_undegraded":"2023-10-07T23:16:28.001591+0000","last_fullsized":"2023-10-07T23:16:28.001591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:35.919781+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:16:35.919781+0000","last_peered":"2023-10-07T23:16:35.919781+0000","last_clean":"2023-10-07T23:16:35.919781+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:16:35.919781+0000","last_undegraded":"2023-10-07T23:16:35.919781+0000","last_fullsized":"2023-10-07T23:16:35.919781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003060+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:16:28.003060+0000","last_peered":"2023-10-07T23:16:28.003060+0000","last_clean":"2023-10-07T23:16:28.003060+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:16:28.003060+0000","last_undegraded":"2023-10-07T23:16:28.003060+0000","last_fullsized":"2023-10-07T23:16:28.003060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999866+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:16:27.999866+0000","last_peered":"2023-10-07T23:16:27.999866+0000","last_clean":"2023-10-07T23:16:27.999866+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:16:27.999866+0000","last_undegraded":"2023-10-07T23:16:27.999866+0000","last_fullsized":"2023-10-07T23:16:27.999866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004005+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:16:28.004005+0000","last_peered":"2023-10-07T23:16:28.004005+0000","last_clean":"2023-10-07T23:16:28.004005+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:16:28.004005+0000","last_undegraded":"2023-10-07T23:16:28.004005+0000","last_fullsized":"2023-10-07T23:16:28.004005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:31.894890+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:16:31.894890+0000","last_peered":"2023-10-07T23:16:31.894890+0000","last_clean":"2023-10-07T23:16:31.894890+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:16:31.894890+0000","last_undegraded":"2023-10-07T23:16:31.894890+0000","last_fullsized":"2023-10-07T23:16:31.894890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007777+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:16:28.007777+0000","last_peered":"2023-10-07T23:16:28.007777+0000","last_clean":"2023-10-07T23:16:28.007777+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:16:28.007777+0000","last_undegraded":"2023-10-07T23:16:28.007777+0000","last_fullsized":"2023-10-07T23:16:28.007777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008687+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:16:28.008687+0000","last_peered":"2023-10-07T23:16:28.008687+0000","last_clean":"2023-10-07T23:16:28.008687+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:16:28.008687+0000","last_undegraded":"2023-10-07T23:16:28.008687+0000","last_fullsized":"2023-10-07T23:16:28.008687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999386+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:16:27.999386+0000","last_peered":"2023-10-07T23:16:27.999386+0000","last_clean":"2023-10-07T23:16:27.999386+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:16:27.999386+0000","last_undegraded":"2023-10-07T23:16:27.999386+0000","last_fullsized":"2023-10-07T23:16:27.999386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007059+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:16:28.007059+0000","last_peered":"2023-10-07T23:16:28.007059+0000","last_clean":"2023-10-07T23:16:28.007059+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:16:28.007059+0000","last_undegraded":"2023-10-07T23:16:28.007059+0000","last_fullsized":"2023-10-07T23:16:28.007059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005716+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:16:28.005716+0000","last_peered":"2023-10-07T23:16:28.005716+0000","last_clean":"2023-10-07T23:16:28.005716+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:16:28.005716+0000","last_undegraded":"2023-10-07T23:16:28.005716+0000","last_fullsized":"2023-10-07T23:16:28.005716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999364+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:16:27.999364+0000","last_peered":"2023-10-07T23:16:27.999364+0000","last_clean":"2023-10-07T23:16:27.999364+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:16:27.999364+0000","last_undegraded":"2023-10-07T23:16:27.999364+0000","last_fullsized":"2023-10-07T23:16:27.999364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.015552+0000","last_change":"2023-10-07T23:16:29.015552+0000","last_active":"2023-10-07T23:16:29.015552+0000","last_peered":"2023-10-07T23:16:29.015552+0000","last_clean":"2023-10-07T23:16:29.015552+0000","last_became_active":"2023-10-07T23:16:29.010895+0000","last_became_peered":"2023-10-07T23:16:29.010895+0000","last_unstale":"2023-10-07T23:16:29.015552+0000","last_undegraded":"2023-10-07T23:16:29.015552+0000","last_fullsized":"2023-10-07T23:16:29.015552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:40.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.012361+0000","last_change":"2023-10-07T23:16:29.012361+0000","last_active":"2023-10-07T23:16:29.012361+0000","last_peered":"2023-10-07T23:16:29.012361+0000","last_clean":"2023-10-07T23:16:29.012361+0000","last_became_active":"2023-10-07T23:16:29.011918+0000","last_became_peered":"2023-10-07T23:16:29.011918+0000","last_unstale":"2023-10-07T23:16:29.012361+0000","last_undegraded":"2023-10-07T23:16:29.012361+0000","last_fullsized":"2023-10-07T23:16:29.012361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:32.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009934+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:16:28.009934+0000","last_peered":"2023-10-07T23:16:28.009934+0000","last_clean":"2023-10-07T23:16:28.009934+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:16:28.009934+0000","last_undegraded":"2023-10-07T23:16:28.009934+0000","last_fullsized":"2023-10-07T23:16:28.009934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001743+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:16:28.001743+0000","last_peered":"2023-10-07T23:16:28.001743+0000","last_clean":"2023-10-07T23:16:28.001743+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.001743+0000","last_undegraded":"2023-10-07T23:16:28.001743+0000","last_fullsized":"2023-10-07T23:16:28.001743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016218+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:16:28.016218+0000","last_peered":"2023-10-07T23:16:28.016218+0000","last_clean":"2023-10-07T23:16:28.016218+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:16:28.016218+0000","last_undegraded":"2023-10-07T23:16:28.016218+0000","last_fullsized":"2023-10-07T23:16:28.016218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.018993+0000","last_change":"2023-10-07T23:16:29.018993+0000","last_active":"2023-10-07T23:16:29.018993+0000","last_peered":"2023-10-07T23:16:29.018993+0000","last_clean":"2023-10-07T23:16:29.018993+0000","last_became_active":"2023-10-07T23:16:29.008350+0000","last_became_peered":"2023-10-07T23:16:29.008350+0000","last_unstale":"2023-10-07T23:16:29.018993+0000","last_undegraded":"2023-10-07T23:16:29.018993+0000","last_fullsized":"2023-10-07T23:16:29.018993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:16.236418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016364+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:16:28.016364+0000","last_peered":"2023-10-07T23:16:28.016364+0000","last_clean":"2023-10-07T23:16:28.016364+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:16:28.016364+0000","last_undegraded":"2023-10-07T23:16:28.016364+0000","last_fullsized":"2023-10-07T23:16:28.016364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006054+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:16:28.006054+0000","last_peered":"2023-10-07T23:16:28.006054+0000","last_clean":"2023-10-07T23:16:28.006054+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:16:28.006054+0000","last_undegraded":"2023-10-07T23:16:28.006054+0000","last_fullsized":"2023-10-07T23:16:28.006054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999879+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:16:27.999879+0000","last_peered":"2023-10-07T23:16:27.999879+0000","last_clean":"2023-10-07T23:16:27.999879+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:16:27.999879+0000","last_undegraded":"2023-10-07T23:16:27.999879+0000","last_fullsized":"2023-10-07T23:16:27.999879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012811+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:16:28.012811+0000","last_peered":"2023-10-07T23:16:28.012811+0000","last_clean":"2023-10-07T23:16:28.012811+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:16:28.012811+0000","last_undegraded":"2023-10-07T23:16:28.012811+0000","last_fullsized":"2023-10-07T23:16:28.012811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999561+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:16:27.999561+0000","last_peered":"2023-10-07T23:16:27.999561+0000","last_clean":"2023-10-07T23:16:27.999561+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:16:27.999561+0000","last_undegraded":"2023-10-07T23:16:27.999561+0000","last_fullsized":"2023-10-07T23:16:27.999561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005469+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:16:28.005469+0000","last_peered":"2023-10-07T23:16:28.005469+0000","last_clean":"2023-10-07T23:16:28.005469+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:16:28.005469+0000","last_undegraded":"2023-10-07T23:16:28.005469+0000","last_fullsized":"2023-10-07T23:16:28.005469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011194+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:16:28.011194+0000","last_peered":"2023-10-07T23:16:28.011194+0000","last_clean":"2023-10-07T23:16:28.011194+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:16:28.011194+0000","last_undegraded":"2023-10-07T23:16:28.011194+0000","last_fullsized":"2023-10-07T23:16:28.011194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006670+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:16:28.006670+0000","last_peered":"2023-10-07T23:16:28.006670+0000","last_clean":"2023-10-07T23:16:28.006670+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:16:28.006670+0000","last_undegraded":"2023-10-07T23:16:28.006670+0000","last_fullsized":"2023-10-07T23:16:28.006670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008316+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:16:28.008316+0000","last_peered":"2023-10-07T23:16:28.008316+0000","last_clean":"2023-10-07T23:16:28.008316+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:16:28.008316+0000","last_undegraded":"2023-10-07T23:16:28.008316+0000","last_fullsized":"2023-10-07T23:16:28.008316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002767+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:16:28.002767+0000","last_peered":"2023-10-07T23:16:28.002767+0000","last_clean":"2023-10-07T23:16:28.002767+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:16:28.002767+0000","last_undegraded":"2023-10-07T23:16:28.002767+0000","last_fullsized":"2023-10-07T23:16:28.002767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999422+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:16:27.999422+0000","last_peered":"2023-10-07T23:16:27.999422+0000","last_clean":"2023-10-07T23:16:27.999422+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:16:27.999422+0000","last_undegraded":"2023-10-07T23:16:27.999422+0000","last_fullsized":"2023-10-07T23:16:27.999422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006299+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:16:28.006299+0000","last_peered":"2023-10-07T23:16:28.006299+0000","last_clean":"2023-10-07T23:16:28.006299+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:16:28.006299+0000","last_undegraded":"2023-10-07T23:16:28.006299+0000","last_fullsized":"2023-10-07T23:16:28.006299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014270+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:16:28.014270+0000","last_peered":"2023-10-07T23:16:28.014270+0000","last_clean":"2023-10-07T23:16:28.014270+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:16:28.014270+0000","last_undegraded":"2023-10-07T23:16:28.014270+0000","last_fullsized":"2023-10-07T23:16:28.014270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999515+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:16:27.999515+0000","last_peered":"2023-10-07T23:16:27.999515+0000","last_clean":"2023-10-07T23:16:27.999515+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:16:27.999515+0000","last_undegraded":"2023-10-07T23:16:27.999515+0000","last_fullsized":"2023-10-07T23:16:27.999515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.049214+0000","last_change":"2023-10-07T23:16:29.049214+0000","last_active":"2023-10-07T23:16:29.049214+0000","last_peered":"2023-10-07T23:16:29.049214+0000","last_clean":"2023-10-07T23:16:29.049214+0000","last_became_active":"2023-10-07T23:16:29.044778+0000","last_became_peered":"2023-10-07T23:16:29.044778+0000","last_unstale":"2023-10-07T23:16:29.049214+0000","last_undegraded":"2023-10-07T23:16:29.049214+0000","last_fullsized":"2023-10-07T23:16:29.049214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:38.061993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007511+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:16:28.007511+0000","last_peered":"2023-10-07T23:16:28.007511+0000","last_clean":"2023-10-07T23:16:28.007511+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:16:28.007511+0000","last_undegraded":"2023-10-07T23:16:28.007511+0000","last_fullsized":"2023-10-07T23:16:28.007511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005590+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:16:28.005590+0000","last_peered":"2023-10-07T23:16:28.005590+0000","last_clean":"2023-10-07T23:16:28.005590+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:16:28.005590+0000","last_undegraded":"2023-10-07T23:16:28.005590+0000","last_fullsized":"2023-10-07T23:16:28.005590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:32.915735+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:16:32.915735+0000","last_peered":"2023-10-07T23:16:32.915735+0000","last_clean":"2023-10-07T23:16:32.915735+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:16:32.915735+0000","last_undegraded":"2023-10-07T23:16:32.915735+0000","last_fullsized":"2023-10-07T23:16:32.915735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007939+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:16:28.007939+0000","last_peered":"2023-10-07T23:16:28.007939+0000","last_clean":"2023-10-07T23:16:28.007939+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:16:28.007939+0000","last_undegraded":"2023-10-07T23:16:28.007939+0000","last_fullsized":"2023-10-07T23:16:28.007939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012043+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:16:28.012043+0000","last_peered":"2023-10-07T23:16:28.012043+0000","last_clean":"2023-10-07T23:16:28.012043+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:16:28.012043+0000","last_undegraded":"2023-10-07T23:16:28.012043+0000","last_fullsized":"2023-10-07T23:16:28.012043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005371+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:16:28.005371+0000","last_peered":"2023-10-07T23:16:28.005371+0000","last_clean":"2023-10-07T23:16:28.005371+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:16:28.005371+0000","last_undegraded":"2023-10-07T23:16:28.005371+0000","last_fullsized":"2023-10-07T23:16:28.005371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007323+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:16:28.007323+0000","last_peered":"2023-10-07T23:16:28.007323+0000","last_clean":"2023-10-07T23:16:28.007323+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:16:28.007323+0000","last_undegraded":"2023-10-07T23:16:28.007323+0000","last_fullsized":"2023-10-07T23:16:28.007323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003056+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:16:28.003056+0000","last_peered":"2023-10-07T23:16:28.003056+0000","last_clean":"2023-10-07T23:16:28.003056+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:16:28.003056+0000","last_undegraded":"2023-10-07T23:16:28.003056+0000","last_fullsized":"2023-10-07T23:16:28.003056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.051993+0000","last_change":"2023-10-07T23:16:29.051993+0000","last_active":"2023-10-07T23:16:29.051993+0000","last_peered":"2023-10-07T23:16:29.051993+0000","last_clean":"2023-10-07T23:16:29.051993+0000","last_became_active":"2023-10-07T23:16:29.046123+0000","last_became_peered":"2023-10-07T23:16:29.046123+0000","last_unstale":"2023-10-07T23:16:29.051993+0000","last_undegraded":"2023-10-07T23:16:29.051993+0000","last_fullsized":"2023-10-07T23:16:29.051993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:52.174959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008468+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:16:28.008468+0000","last_peered":"2023-10-07T23:16:28.008468+0000","last_clean":"2023-10-07T23:16:28.008468+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:16:28.008468+0000","last_undegraded":"2023-10-07T23:16:28.008468+0000","last_fullsized":"2023-10-07T23:16:28.008468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002713+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:16:28.002713+0000","last_peered":"2023-10-07T23:16:28.002713+0000","last_clean":"2023-10-07T23:16:28.002713+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:16:28.002713+0000","last_undegraded":"2023-10-07T23:16:28.002713+0000","last_fullsized":"2023-10-07T23:16:28.002713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006794+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:16:28.006794+0000","last_peered":"2023-10-07T23:16:28.006794+0000","last_clean":"2023-10-07T23:16:28.006794+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:16:28.006794+0000","last_undegraded":"2023-10-07T23:16:28.006794+0000","last_fullsized":"2023-10-07T23:16:28.006794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006214+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:16:28.006214+0000","last_peered":"2023-10-07T23:16:28.006214+0000","last_clean":"2023-10-07T23:16:28.006214+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:16:28.006214+0000","last_undegraded":"2023-10-07T23:16:28.006214+0000","last_fullsized":"2023-10-07T23:16:28.006214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012237+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:16:28.012237+0000","last_peered":"2023-10-07T23:16:28.012237+0000","last_clean":"2023-10-07T23:16:28.012237+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:16:28.012237+0000","last_undegraded":"2023-10-07T23:16:28.012237+0000","last_fullsized":"2023-10-07T23:16:28.012237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044932+0000","last_change":"2023-10-07T23:16:29.044932+0000","last_active":"2023-10-07T23:16:29.044932+0000","last_peered":"2023-10-07T23:16:29.044932+0000","last_clean":"2023-10-07T23:16:29.044932+0000","last_became_active":"2023-10-07T23:16:29.044454+0000","last_became_peered":"2023-10-07T23:16:29.044454+0000","last_unstale":"2023-10-07T23:16:29.044932+0000","last_undegraded":"2023-10-07T23:16:29.044932+0000","last_fullsized":"2023-10-07T23:16:29.044932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:55.975049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003230+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:16:28.003230+0000","last_peered":"2023-10-07T23:16:28.003230+0000","last_clean":"2023-10-07T23:16:28.003230+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:16:28.003230+0000","last_undegraded":"2023-10-07T23:16:28.003230+0000","last_fullsized":"2023-10-07T23:16:28.003230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008782+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:16:28.008782+0000","last_peered":"2023-10-07T23:16:28.008782+0000","last_clean":"2023-10-07T23:16:28.008782+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:16:28.008782+0000","last_undegraded":"2023-10-07T23:16:28.008782+0000","last_fullsized":"2023-10-07T23:16:28.008782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015307+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:16:28.015307+0000","last_peered":"2023-10-07T23:16:28.015307+0000","last_clean":"2023-10-07T23:16:28.015307+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:16:28.015307+0000","last_undegraded":"2023-10-07T23:16:28.015307+0000","last_fullsized":"2023-10-07T23:16:28.015307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003934+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:16:28.003934+0000","last_peered":"2023-10-07T23:16:28.003934+0000","last_clean":"2023-10-07T23:16:28.003934+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:16:28.003934+0000","last_undegraded":"2023-10-07T23:16:28.003934+0000","last_fullsized":"2023-10-07T23:16:28.003934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008400+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:16:28.008400+0000","last_peered":"2023-10-07T23:16:28.008400+0000","last_clean":"2023-10-07T23:16:28.008400+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:16:28.008400+0000","last_undegraded":"2023-10-07T23:16:28.008400+0000","last_fullsized":"2023-10-07T23:16:28.008400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005732+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:16:28.005732+0000","last_peered":"2023-10-07T23:16:28.005732+0000","last_clean":"2023-10-07T23:16:28.005732+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:16:28.005732+0000","last_undegraded":"2023-10-07T23:16:28.005732+0000","last_fullsized":"2023-10-07T23:16:28.005732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000267+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:16:28.000267+0000","last_peered":"2023-10-07T23:16:28.000267+0000","last_clean":"2023-10-07T23:16:28.000267+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:16:28.000267+0000","last_undegraded":"2023-10-07T23:16:28.000267+0000","last_fullsized":"2023-10-07T23:16:28.000267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003067+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:16:28.003067+0000","last_peered":"2023-10-07T23:16:28.003067+0000","last_clean":"2023-10-07T23:16:28.003067+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:16:28.003067+0000","last_undegraded":"2023-10-07T23:16:28.003067+0000","last_fullsized":"2023-10-07T23:16:28.003067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9805,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":453,"num_read_kb":242,"num_write":209,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":29415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383705,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":495361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.746},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.71599999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0189999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.76000000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.82699999999999996}]},{"osd":9,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.89900000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.94999999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298908,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446244,"statfs":{"total":95995035648,"available":95688953856,"internally_reserved":0,"allocated":1658880,"data_stored":1089665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.45000000000000001,"15min":0.45200000000000001},"min":{"1min":0.31,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.504}]},{"osd":2,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.47099999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66100000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.83799999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72499999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51700000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65400000000000003,"15min":0.65800000000000003},"min":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81699999999999995,"5min":1.159,"15min":1.159},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.86299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.746},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0960000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1459999999999999,"5min":1.214,"15min":1.214},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.62,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.91100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89600000000000002,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63500000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98199999999999998,"5min":1.171,"15min":1.171},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.91400000000000003,"15min":0.94199999999999995},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.93700000000000006,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.71799999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.70599999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68200000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84199999999999997,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.88,"5min":1.054,"15min":1.054},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86099999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.84699999999999998,"15min":0.86099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.325,"15min":1.325},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95999999999999996,"5min":1.278,"15min":1.278},"last":0.71499999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.67600000000000005,"15min":0.69799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76700000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96499999999999997,"5min":1.385,"15min":1.385},"last":0.61099999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.52800000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97799999999999998,"5min":1.345,"15min":1.345},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.84799999999999998,"15min":0.873},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.82299999999999995,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.90100000000000002,"15min":0.92700000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97599999999999998,"5min":1.579,"15min":1.579},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.96399999999999997,"15min":0.99299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0649999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.113,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.018,"15min":1.0429999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2310000000000001,"5min":1.603,"15min":1.603},"last":0.501}]},{"osd":8,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.026,"15min":1.056},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.038,"5min":1.782,"15min":1.782},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.96599999999999997,"15min":0.998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0529999999999999,"5min":1.956,"15min":1.956},"last":0.625}]},{"osd":10,"last update":"Sat Oct 7 23:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.008,"15min":1.0249999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2849999999999999,"5min":1.647,"15min":1.647},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.0009999999999999,"15min":1.0189999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.165,"5min":1.663,"15min":1.663},"last":0.59299999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.747},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68500000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":1044480,"data_stored":495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39700000000000002}]},{"osd":3,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75}]},{"osd":4,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.496}]},{"osd":5,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.434},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.59299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.022,"5min":1.597,"15min":1.597},"last":0.72199999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1,"15min":1},"last":0.55400000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.59899999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92900000000000005,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96399999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.72699999999999998,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.61899999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298888,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446264,"statfs":{"total":95995035648,"available":95688974336,"internally_reserved":0,"allocated":1638400,"data_stored":1087555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57999999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.31900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60699999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.56000000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81899999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":1.51,"15min":1.51},"last":0.72299999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82599999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66300000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48699999999999999,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68000000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.29099999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63200000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.754}]},{"osd":5,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84699999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.48299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.88500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.92800000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.93100000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78100000000000003,"15min":0.79100000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.03,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90100000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83399999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98399999999999999,"5min":1.056,"15min":1.056},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0029999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.89700000000000002}]},{"osd":11,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.94299999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":494596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70999999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53800000000000003,"15min":0.52500000000000002},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70999999999999996,"5min":0.878,"15min":0.878},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.70999999999999996,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.5700000000000003,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.50600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.56299999999999994,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.879}]},{"osd":3,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.877,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69199999999999995,"15min":0.70699999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.95599999999999996,"5min":1.014,"15min":1.014},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75,"15min":0.751},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.746}]},{"osd":5,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.099,"15min":1.099},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79700000000000004,"15min":0.77900000000000003},"min":{"1min":0.53100000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.875,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0700000000000001,"5min":6.524,"15min":6.524},"last":0.76500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85799999999999998,"15min":0.85599999999999998},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.0509999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88800000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0580000000000001,"5min":1.165,"15min":1.165},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Oct 7 23:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.1619999999999999,"15min":1.206},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1679999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.113,"15min":1.1459999999999999},"min":{"1min":0.64000000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.472,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.56899999999999995,"15min":0.52600000000000002},"min":{"1min":0.434,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64800000000000002}]},{"osd":3,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.80500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46600000000000003,"15min":0.42599999999999999},"min":{"1min":0.28499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.60799999999999998,"15min":0.54800000000000004},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.374,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67500000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.51700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72899999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58499999999999996}]},{"osd":8,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.75900000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.751,"15min":0.754},"min":{"1min":0.56999999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.89200000000000002}]},{"osd":11,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.875},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.91700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":1.022,"15min":1.022},"last":0.81799999999999995}]},{"osd":2,"last update":"Sat Oct 7 23:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.022,"5min":1.212,"15min":1.212},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.48199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69499999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.73999999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.71799999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.52100000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.94699999999999995,"5min":1.155,"15min":1.155},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96299999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78100000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.79500000000000004}]},{"osd":9,"last update":"Sat Oct 7 23:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91800000000000004,"15min":0.92600000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2150000000000001,"5min":1.323,"15min":1.323},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.86299999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.873}]},{"osd":10,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88800000000000001,"15min":0.88900000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.95399999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.98699999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446836,"statfs":{"total":95995035648,"available":95689560064,"internally_reserved":0,"allocated":1052672,"data_stored":499485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54500000000000004,"15min":0.499},"min":{"1min":0.436,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.51000000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.754,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84999999999999998,"5min":0.871,"15min":0.871},"last":0.61299999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.74099999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.47299999999999998,"15min":0.44500000000000001},"min":{"1min":0.38300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.74199999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.36299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.40600000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67500000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72799999999999998}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.628,"15min":0.60099999999999998},"min":{"1min":0.59099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.55800000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.65100000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.72999999999999998,"15min":0.70499999999999996},"min":{"1min":0.57699999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.747,"15min":0.72199999999999998},"min":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.79000000000000004}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76800000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":1040384,"data_stored":494675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.46999999999999997,"15min":0.41599999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.51800000000000002,"15min":0.45200000000000001},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.755}]},{"osd":1,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.60299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89500000000000002,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.85699999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.69599999999999995,"15min":0.625},"min":{"1min":0.46800000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69099999999999995,"15min":0.622},"min":{"1min":0.56100000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.95099999999999996,"5min":1.218,"15min":1.218},"last":1.0680000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.68300000000000005,"15min":0.629},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.82699999999999996}]},{"osd":5,"last update":"Sat Oct 7 23:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.71699999999999997,"15min":0.67700000000000005},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.0069999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70899999999999996,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1459999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.96799999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.374,"15min":0.374},"max":{"1min":1.048,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Sat Oct 7 23:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.69899999999999995},"min":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.72099999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.089,"5min":1.335,"15min":1.335},"last":0.98699999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.71399999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2230000000000001,"5min":1.288,"15min":1.288},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.0029999999999999,"15min":1.0189999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.169,"5min":1.51,"15min":1.51},"last":0.78600000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0920000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.876},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.95299999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.036},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.0509999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":494897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.36499999999999999},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.38800000000000001,"15min":0.35299999999999998},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58799999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.624},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.49399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84499999999999997,"5min":0.88,"15min":0.88},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.53300000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":1.034},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.46999999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77200000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.55300000000000005},"min":{"1min":0.52500000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90300000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60599999999999998,"15min":0.55400000000000005},"min":{"1min":0.58399999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.80800000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.55500000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.65600000000000003,"15min":0.63700000000000001},"min":{"1min":0.59599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.75900000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.55700000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.65500000000000003,"15min":0.621},"min":{"1min":0.61599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86899999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.92400000000000004}]},{"osd":9,"last update":"Sat Oct 7 23:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.80200000000000005,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.90600000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.79800000000000004}]},{"osd":10,"last update":"Sat Oct 7 23:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.752,"15min":0.749},"min":{"1min":0.66100000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89100000000000001,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81799999999999995},"min":{"1min":0.73699999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":1.0109999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.98799999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446256,"statfs":{"total":95995035648,"available":95688966144,"internally_reserved":0,"allocated":1646592,"data_stored":1086978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.47999999999999998},"min":{"1min":0.30299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.48399999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48099999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53700000000000003,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.497},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60399999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.39300000000000002,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.376},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40699999999999997,"15min":0.378},"min":{"1min":0.27000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.60799999999999998,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.60299999999999998,"15min":0.54500000000000004},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58199999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.621},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59199999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89600000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.745}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.47899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55100000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1,"5min":1,"15min":1},"last":0.63500000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93899999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:16:40.348 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-07T23:16:40.349 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-07T23:16:40.349 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-07T23:16:40.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-07T23:16:41.121 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:41.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:41 smithi067 ceph-mon[82723]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:16:41.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:41 smithi067 ceph-mon[82723]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:41.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:41 smithi123 ceph-mon[122926]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:16:41.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:41 smithi123 ceph-mon[122926]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:41.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:41 smithi086 ceph-mon[122131]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:16:41.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:41 smithi086 ceph-mon[122131]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 1 objects/s recovering 2023-10-07T23:16:42.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 -- 172.21.15.67:0/2179461538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 msgr2=0x7f2924103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:42.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2179461538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924103230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f29200049f0 tx=0x7f2920009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:42.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 -- 172.21.15.67:0/2179461538 shutdown_connections 2023-10-07T23:16:42.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2179461538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29240fd550 0x7f2924109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2179461538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924103230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2179461538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 0x7f29240fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 -- 172.21.15.67:0/2179461538 >> 172.21.15.67:0/2179461538 conn(0x7f2924074320 msgr2=0x7f2924074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 -- 172.21.15.67:0/2179461538 shutdown_connections 2023-10-07T23:16:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.633+0000 7f292b8f2700 1 -- 172.21.15.67:0/2179461538 wait complete. 2023-10-07T23:16:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.634+0000 7f292b8f2700 1 Processor -- start 2023-10-07T23:16:42.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.634+0000 7f292b8f2700 1 -- start start 2023-10-07T23:16:42.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292b8f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29240fd550 0x7f2924113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:42.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292b8f2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:42.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292b8f2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 0x7f292410dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:42.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292b8f2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292410e1e0 con 0x7f2924102e60 2023-10-07T23:16:42.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292b8f2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292410e350 con 0x7f2924103800 2023-10-07T23:16:42.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292b8f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292410e510 con 0x7f29240fd550 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44138/0 (socket says 172.21.15.67:44138) 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 -- 172.21.15.67:0/2407247589 learned_addr learned my addr 172.21.15.67:0/2407247589 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2929e8f700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 0x7f292410dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f292968e700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29240fd550 0x7f2924113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 -- 172.21.15.67:0/2407247589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29240fd550 msgr2=0x7f2924113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:42.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29240fd550 0x7f2924113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 -- 172.21.15.67:0/2407247589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 msgr2=0x7f292410dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 0x7f292410dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2928e8d700 1 -- 172.21.15.67:0/2407247589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920005040 con 0x7f2924102e60 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.635+0000 7f2929e8f700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 0x7f292410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.636+0000 7f2928e8d700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924114050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f291400ebc0 tx=0x7f291400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.636+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914007cc0 con 0x7f2924102e60 2023-10-07T23:16:42.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.636+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291400f040 con 0x7f2924102e60 2023-10-07T23:16:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.636+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292410e790 con 0x7f2924102e60 2023-10-07T23:16:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.636+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291400c990 con 0x7f2924102e60 2023-10-07T23:16:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.636+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292411ab00 con 0x7f2924102e60 2023-10-07T23:16:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.637+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2924063f20 con 0x7f2924102e60 2023-10-07T23:16:42.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.639+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f291400cd40 con 0x7f2924102e60 2023-10-07T23:16:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.643+0000 7f2919ffb700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f291009b7f0 0x7f291009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.643+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f29140c4c30 con 0x7f2924102e60 2023-10-07T23:16:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.643+0000 7f292968e700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f291009b7f0 0x7f291009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:42.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.644+0000 7f292968e700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f291009b7f0 0x7f291009dca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f292000b090 tx=0x7f2920009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:42.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.645+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f291404da00 con 0x7f2924102e60 2023-10-07T23:16:42.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.894+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f292410cce0 con 0x7f2924102e60 2023-10-07T23:16:42.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.895+0000 7f2919ffb700 1 -- 172.21.15.67:0/2407247589 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f29140900e0 con 0x7f2924102e60 2023-10-07T23:16:42.896 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:42.896 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f291009b7f0 msgr2=0x7f291009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f291009b7f0 0x7f291009dca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f292000b090 tx=0x7f2920009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 msgr2=0x7f2924114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924114050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f291400ebc0 tx=0x7f291400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 shutdown_connections 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924103800 0x7f292410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f291009b7f0 0x7f291009dca0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29240fd550 0x7f2924113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 --2- 172.21.15.67:0/2407247589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2924102e60 0x7f2924114050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:42.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 >> 172.21.15.67:0/2407247589 conn(0x7f2924074320 msgr2=0x7f2924100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:42.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.897+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 shutdown_connections 2023-10-07T23:16:42.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:42.898+0000 7f292b8f2700 1 -- 172.21.15.67:0/2407247589 wait complete. 2023-10-07T23:16:43.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:43 smithi067 ceph-mon[82723]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:43.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:43 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2407247589' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-07T23:16:43.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:43 smithi123 ceph-mon[122926]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:43.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:43 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2407247589' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-07T23:16:43.404 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-07T23:16:43.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:16:43.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:43 smithi086 ceph-mon[122131]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:43.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:43 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2407247589' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 -- 172.21.15.67:0/4013550998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458070570 msgr2=0x7f04580709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 --2- 172.21.15.67:0/4013550998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458070570 0x7f04580709c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f044c0049c0 tx=0x7f044c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 -- 172.21.15.67:0/4013550998 shutdown_connections 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 --2- 172.21.15.67:0/4013550998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0458100780 0x7f045810b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 --2- 172.21.15.67:0/4013550998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0458071bc0 0x7f0458070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 --2- 172.21.15.67:0/4013550998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458070570 0x7f04580709c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 -- 172.21.15.67:0/4013550998 >> 172.21.15.67:0/4013550998 conn(0x7f04580f8410 msgr2=0x7f04580fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 -- 172.21.15.67:0/4013550998 shutdown_connections 2023-10-07T23:16:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.598+0000 7f0457fff700 1 -- 172.21.15.67:0/4013550998 wait complete. 2023-10-07T23:16:43.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 Processor -- start 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 -- start start 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0458070570 0x7f0458113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 0x7f0458114520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0458100780 0x7f045811b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045811bbc0 con 0x7f0458071bc0 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045811bd30 con 0x7f0458100780 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.599+0000 7f0457fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045811bef0 con 0x7f0458070570 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 0x7f0458114520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 0x7f0458114520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44164/0 (socket says 172.21.15.67:44164) 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 -- 172.21.15.67:0/1456699624 learned_addr learned my addr 172.21.15.67:0/1456699624 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f0456ffd700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0458070570 0x7f0458113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04577fe700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0458100780 0x7f045811b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 -- 172.21.15.67:0/1456699624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0458070570 msgr2=0x7f0458113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0458070570 0x7f0458113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 -- 172.21.15.67:0/1456699624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0458100780 msgr2=0x7f045811b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0458100780 0x7f045811b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 -- 172.21.15.67:0/1456699624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c005040 con 0x7f0458071bc0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f04567fc700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 0x7f0458114520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f044c009180 tx=0x7f044c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.600+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c020070 con 0x7f0458071bc0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.601+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045811c170 con 0x7f0458071bc0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.601+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f044c009520 con 0x7f0458071bc0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.601+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00e070 con 0x7f0458071bc0 2023-10-07T23:16:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.601+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045811c5f0 con 0x7f0458071bc0 2023-10-07T23:16:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.602+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04580fed80 con 0x7f0458071bc0 2023-10-07T23:16:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.603+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f044c009690 con 0x7f0458071bc0 2023-10-07T23:16:43.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.606+0000 7f043f7fe700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f044009b770 0x7f044009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:43.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.606+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f044c0c6390 con 0x7f0458071bc0 2023-10-07T23:16:43.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.606+0000 7f0456ffd700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f044009b770 0x7f044009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:43.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.607+0000 7f0456ffd700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f044009b770 0x7f044009dc20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0448009b80 tx=0x7f044800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:43.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.610+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f044c0918a0 con 0x7f0458071bc0 2023-10-07T23:16:43.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.816+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f045802b330 con 0x7f0458071bc0 2023-10-07T23:16:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.818+0000 7f043f7fe700 1 -- 172.21.15.67:0/1456699624 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7f044c091030 con 0x7f0458071bc0 2023-10-07T23:16:43.838 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:43.839 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:16:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f044009b770 msgr2=0x7f044009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f044009b770 0x7f044009dc20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0448009b80 tx=0x7f044800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 msgr2=0x7f0458114520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 0x7f0458114520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f044c009180 tx=0x7f044c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 shutdown_connections 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0458100780 0x7f045811b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f044009b770 0x7f044009dc20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0458070570 0x7f0458113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 --2- 172.21.15.67:0/1456699624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0458071bc0 0x7f0458114520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 >> 172.21.15.67:0/1456699624 conn(0x7f04580f8410 msgr2=0x7f04580fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 shutdown_connections 2023-10-07T23:16:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:43.820+0000 7f0457fff700 1 -- 172.21.15.67:0/1456699624 wait complete. 2023-10-07T23:16:43.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:15:25.227955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:15:26.323230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-07T23:15:29.861861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:16:43.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 -- 172.21.15.67:0/649781168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 msgr2=0x7f87f40ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 --2- 172.21.15.67:0/649781168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f40ffc30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f87f0004900 tx=0x7f87f0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 -- 172.21.15.67:0/649781168 shutdown_connections 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 --2- 172.21.15.67:0/649781168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f4102a90 0x7f87f4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 --2- 172.21.15.67:0/649781168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f40ffc30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 --2- 172.21.15.67:0/649781168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 0x7f87f4102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 -- 172.21.15.67:0/649781168 >> 172.21.15.67:0/649781168 conn(0x7f87f40f7370 msgr2=0x7f87f40f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:44.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 -- 172.21.15.67:0/649781168 shutdown_connections 2023-10-07T23:16:44.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.036+0000 7f87fa39d700 1 -- 172.21.15.67:0/649781168 wait complete. 2023-10-07T23:16:44.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 Processor -- start 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 -- start start 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 0x7f87f410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f4102a90 0x7f87f4114f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f4115590 con 0x7f87f40fd850 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f4115700 con 0x7f87f4100170 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.037+0000 7f87fa39d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f41158c0 con 0x7f87f4102a90 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44188/0 (socket says 172.21.15.67:44188) 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 -- 172.21.15.67:0/4252094563 learned_addr learned my addr 172.21.15.67:0/4252094563 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f9b9c700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f4102a90 0x7f87f4114f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f8b9a700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 0x7f87f410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 -- 172.21.15.67:0/4252094563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f4102a90 msgr2=0x7f87f4114f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f4102a90 0x7f87f4114f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 -- 172.21.15.67:0/4252094563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 msgr2=0x7f87f410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 0x7f87f410def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 -- 172.21.15.67:0/4252094563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4005040 con 0x7f87f40fd850 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f8b9a700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 0x7f87f410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87f939b700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f410d9b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f87f0007f50 tx=0x7f87f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0020070 con 0x7f87f40fd850 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.038+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f0005040 con 0x7f87f40fd850 2023-10-07T23:16:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.039+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f00098a0 con 0x7f87f40fd850 2023-10-07T23:16:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.039+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000e040 con 0x7f87f40fd850 2023-10-07T23:16:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.039+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f4115e90 con 0x7f87f40fd850 2023-10-07T23:16:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.040+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f40f8ea0 con 0x7f87f40fd850 2023-10-07T23:16:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.042+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f87f001c530 con 0x7f87f40fd850 2023-10-07T23:16:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.043+0000 7f87e9ffb700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f87e009b770 0x7f87e009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.044+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f87f00c6320 con 0x7f87f40fd850 2023-10-07T23:16:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.044+0000 7f87f8b9a700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f87e009b770 0x7f87e009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.044+0000 7f87f8b9a700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f87e009b770 0x7f87e009dc20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009fd0 tx=0x7f87e4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.050+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f87f00917d0 con 0x7f87f40fd850 2023-10-07T23:16:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.252+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87f41084f0 con 0x7f87f40fd850 2023-10-07T23:16:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.253+0000 7f87e9ffb700 1 -- 172.21.15.67:0/4252094563 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87f41084f0 con 0x7f87f40fd850 2023-10-07T23:16:44.253 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:16:44.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:44 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1456699624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:44.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f87e009b770 msgr2=0x7f87e009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f87e009b770 0x7f87e009dc20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009fd0 tx=0x7f87e4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 msgr2=0x7f87f410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f410d9b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f87f0007f50 tx=0x7f87f0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 shutdown_connections 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f87e009b770 0x7f87e009dc20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87f4102a90 0x7f87f4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f40fd850 0x7f87f410d9b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 --2- 172.21.15.67:0/4252094563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87f4100170 0x7f87f410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 >> 172.21.15.67:0/4252094563 conn(0x7f87f40f7370 msgr2=0x7f87f40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 shutdown_connections 2023-10-07T23:16:44.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.255+0000 7f87fa39d700 1 -- 172.21.15.67:0/4252094563 wait complete. 2023-10-07T23:16:44.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:16:44.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:44 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1456699624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:44.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:44 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1456699624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 -- 172.21.15.67:0/3386952869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8071b60 msgr2=0x7f86a806ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 --2- 172.21.15.67:0/3386952869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8071b60 0x7f86a806ffd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f869c0049c0 tx=0x7f869c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 -- 172.21.15.67:0/3386952869 shutdown_connections 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 --2- 172.21.15.67:0/3386952869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8071b60 0x7f86a806ffd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 --2- 172.21.15.67:0/3386952869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a8104bf0 0x7f86a8107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 --2- 172.21.15.67:0/3386952869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8070510 0x7f86a8070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 -- 172.21.15.67:0/3386952869 >> 172.21.15.67:0/3386952869 conn(0x7f86a80f8450 msgr2=0x7f86a80fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 -- 172.21.15.67:0/3386952869 shutdown_connections 2023-10-07T23:16:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.467+0000 7f86add9d700 1 -- 172.21.15.67:0/3386952869 wait complete. 2023-10-07T23:16:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 Processor -- start 2023-10-07T23:16:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 -- start start 2023-10-07T23:16:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 0x7f86a810fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a8071b60 0x7f86a81102d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8104bf0 0x7f86a8117360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8117970 con 0x7f86a8104bf0 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8117ae0 con 0x7f86a8070510 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86add9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8117ca0 con 0x7f86a8071b60 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86acd9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 0x7f86a810fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86ad59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8104bf0 0x7f86a8117360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86acd9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 0x7f86a810fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:50408/0 (socket says 172.21.15.67:50408) 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.468+0000 7f86acd9b700 1 -- 172.21.15.67:0/3330532212 learned_addr learned my addr 172.21.15.67:0/3330532212 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86a7fff700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a8071b60 0x7f86a81102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86acd9b700 1 -- 172.21.15.67:0/3330532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a8071b60 msgr2=0x7f86a81102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86acd9b700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a8071b60 0x7f86a81102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86acd9b700 1 -- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8104bf0 msgr2=0x7f86a8117360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86acd9b700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8104bf0 0x7f86a8117360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86acd9b700 1 -- 172.21.15.67:0/3330532212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694005040 con 0x7f86a8070510 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86acd9b700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 0x7f86a810fd90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f869c009180 tx=0x7f869c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c020070 con 0x7f86a8070510 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c005040 con 0x7f86a8070510 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f869c009520 con 0x7f86a8070510 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.469+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c00e070 con 0x7f86a8070510 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.470+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a81181a0 con 0x7f86a8070510 2023-10-07T23:16:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.470+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a810a750 con 0x7f86a8070510 2023-10-07T23:16:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.472+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f869c009690 con 0x7f86a8070510 2023-10-07T23:16:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.474+0000 7f86a57fa700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f869809b770 0x7f869809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.474+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f869c0c6b90 con 0x7f86a8070510 2023-10-07T23:16:44.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.474+0000 7f86a7fff700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f869809b770 0x7f869809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.475+0000 7f86a7fff700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f869809b770 0x7f869809dc20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8694009fd0 tx=0x7f869400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.476+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f869c092040 con 0x7f86a8070510 2023-10-07T23:16:44.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.679+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86a8118330 con 0x7f86a8070510 2023-10-07T23:16:44.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.680+0000 7f86a57fa700 1 -- 172.21.15.67:0/3330532212 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f869c095660 con 0x7f86a8070510 2023-10-07T23:16:44.681 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f869809b770 msgr2=0x7f869809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f869809b770 0x7f869809dc20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8694009fd0 tx=0x7f869400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 msgr2=0x7f86a810fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 0x7f86a810fd90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f869c009180 tx=0x7f869c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 shutdown_connections 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a8070510 0x7f86a810fd90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f869809b770 0x7f869809dc20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a8104bf0 0x7f86a8117360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 --2- 172.21.15.67:0/3330532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86a8071b60 0x7f86a81102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 >> 172.21.15.67:0/3330532212 conn(0x7f86a80f8450 msgr2=0x7f86a8106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:44.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 shutdown_connections 2023-10-07T23:16:44.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.683+0000 7f86add9d700 1 -- 172.21.15.67:0/3330532212 wait complete. 2023-10-07T23:16:44.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:16:44.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 -- 172.21.15.67:0/2637072124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 msgr2=0x7f0fbc0fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 --2- 172.21.15.67:0/2637072124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc0fd100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00049c0 tx=0x7f0fb0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 -- 172.21.15.67:0/2637072124 shutdown_connections 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 --2- 172.21.15.67:0/2637072124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc0fd100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 --2- 172.21.15.67:0/2637072124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fbc0fe890 0x7f0fbc105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 --2- 172.21.15.67:0/2637072124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 0x7f0fbc0fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 -- 172.21.15.67:0/2637072124 >> 172.21.15.67:0/2637072124 conn(0x7f0fbc0f8410 msgr2=0x7f0fbc0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 -- 172.21.15.67:0/2637072124 shutdown_connections 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.894+0000 7f0fc199d700 1 -- 172.21.15.67:0/2637072124 wait complete. 2023-10-07T23:16:44.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 Processor -- start 2023-10-07T23:16:44.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 -- start start 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc10fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 0x7f0fbc1101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fbc0fe890 0x7f0fbc1171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc117830 con 0x7f0fbc0fdf00 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc1179a0 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.895+0000 7f0fc199d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc117b60 con 0x7f0fbc0fe890 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fbbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 0x7f0fbc1101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc10fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc10fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:50420/0 (socket says 172.21.15.67:50420) 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fbbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 0x7f0fbc1101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44228/0 (socket says 172.21.15.67:44228) 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 -- 172.21.15.67:0/752708895 learned_addr learned my addr 172.21.15.67:0/752708895 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc119c700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fbc0fe890 0x7f0fbc1171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 -- 172.21.15.67:0/752708895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fbc0fe890 msgr2=0x7f0fbc1171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fbc0fe890 0x7f0fbc1171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 -- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 msgr2=0x7f0fbc1101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:44.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 0x7f0fbc1101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 -- 172.21.15.67:0/752708895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0005040 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.896+0000 7f0fc099b700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc10fc60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009150 tx=0x7f0fb0009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.897+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0020070 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.897+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb00041a0 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.897+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000e070 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.897+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc117de0 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.897+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc118190 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.898+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc108550 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.900+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f0fb001c430 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.901+0000 7f0fb97fa700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0fac09b7f0 0x7f0fac09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.901+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f0fb00c6950 con 0x7f0fbc0fccf0 2023-10-07T23:16:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.901+0000 7f0fbbfff700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0fac09b7f0 0x7f0fac09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.902+0000 7f0fbbfff700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0fac09b7f0 0x7f0fac09dca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009b80 tx=0x7f0fa800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:44.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:44.903+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f0fb0091e80 con 0x7f0fbc0fccf0 2023-10-07T23:16:45.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.108+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fbc1183b0 con 0x7f0fbc0fccf0 2023-10-07T23:16:45.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.109+0000 7f0fb97fa700 1 -- 172.21.15.67:0/752708895 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fb00954a0 con 0x7f0fbc0fccf0 2023-10-07T23:16:45.110 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-07T23:16:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.111+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0fac09b7f0 msgr2=0x7f0fac09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.111+0000 7f0fc199d700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0fac09b7f0 0x7f0fac09dca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009b80 tx=0x7f0fa800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:45.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.111+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 msgr2=0x7f0fbc10fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.111+0000 7f0fc199d700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc10fc60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009150 tx=0x7f0fb0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:45.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 shutdown_connections 2023-10-07T23:16:45.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc0fccf0 0x7f0fbc10fc60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f0fac09b7f0 0x7f0fac09dca0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fbc0fe890 0x7f0fbc1171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 --2- 172.21.15.67:0/752708895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fbc0fdf00 0x7f0fbc1101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 >> 172.21.15.67:0/752708895 conn(0x7f0fbc0f8410 msgr2=0x7f0fbc0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:45.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 shutdown_connections 2023-10-07T23:16:45.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.112+0000 7f0fc199d700 1 -- 172.21.15.67:0/752708895 wait complete. 2023-10-07T23:16:45.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-07T23:16:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 -- 172.21.15.67:0/309718981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fdf00 msgr2=0x7f6df40fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/309718981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fdf00 0x7f6df40fe350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6de40049c0 tx=0x7f6de4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 -- 172.21.15.67:0/309718981 shutdown_connections 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/309718981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40fccf0 0x7f6df40fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/309718981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fe890 0x7f6df4105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/309718981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fdf00 0x7f6df40fe350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.321+0000 7f6dfab9d700 1 -- 172.21.15.67:0/309718981 >> 172.21.15.67:0/309718981 conn(0x7f6df40f8450 msgr2=0x7f6df40fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.322+0000 7f6dfab9d700 1 -- 172.21.15.67:0/309718981 shutdown_connections 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.322+0000 7f6dfab9d700 1 -- 172.21.15.67:0/309718981 wait complete. 2023-10-07T23:16:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.322+0000 7f6dfab9d700 1 Processor -- start 2023-10-07T23:16:45.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.322+0000 7f6dfab9d700 1 -- start start 2023-10-07T23:16:45.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfab9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 0x7f6df410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfab9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fdf00 0x7f6df410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfab9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40fe890 0x7f6df4114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfab9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4115540 con 0x7f6df40fccf0 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfab9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df41156b0 con 0x7f6df40fe890 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfab9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4115870 con 0x7f6df40fdf00 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 0x7f6df410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 0x7f6df410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44252/0 (socket says 172.21.15.67:44252) 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 -- 172.21.15.67:0/1921547842 learned_addr learned my addr 172.21.15.67:0/1921547842 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6dfa39c700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40fe890 0x7f6df4114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df939a700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fdf00 0x7f6df410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 -- 172.21.15.67:0/1921547842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fdf00 msgr2=0x7f6df410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fdf00 0x7f6df410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 -- 172.21.15.67:0/1921547842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40fe890 msgr2=0x7f6df4114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40fe890 0x7f6df4114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df9b9b700 1 -- 172.21.15.67:0/1921547842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4005040 con 0x7f6df40fccf0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.323+0000 7f6df939a700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fdf00 0x7f6df410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.324+0000 7f6df9b9b700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 0x7f6df410d960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6df000ebf0 tx=0x7f6df000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.324+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0007cc0 con 0x7f6df40fccf0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.324+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df4115af0 con 0x7f6df40fccf0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.324+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6df000f040 con 0x7f6df40fccf0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.324+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0009460 con 0x7f6df40fccf0 2023-10-07T23:16:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.324+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df4116030 con 0x7f6df40fccf0 2023-10-07T23:16:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.325+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df4108510 con 0x7f6df40fccf0 2023-10-07T23:16:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.327+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f6df000c660 con 0x7f6df40fccf0 2023-10-07T23:16:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.328+0000 7f6dea7fc700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6de009b770 0x7f6de009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.328+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f6df00c4da0 con 0x7f6df40fccf0 2023-10-07T23:16:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.328+0000 7f6df939a700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6de009b770 0x7f6de009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.329+0000 7f6df939a700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6de009b770 0x7f6de009dc20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6de4007f50 tx=0x7f6de4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:45.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.332+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f6df004da00 con 0x7f6df40fccf0 2023-10-07T23:16:45.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:45 smithi123 ceph-mon[122926]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:45.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:45 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4252094563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:16:45.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:45 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3330532212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:16:45.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:45 smithi086 ceph-mon[122131]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:45.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:45 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4252094563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:16:45.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:45 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3330532212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:16:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:45 smithi067 ceph-mon[82723]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s; 0 B/s, 0 objects/s recovering 2023-10-07T23:16:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:45 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4252094563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:16:45.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:45 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3330532212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:16:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.532+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6df4116400 con 0x7f6df40fccf0 2023-10-07T23:16:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.533+0000 7f6dea7fc700 1 -- 172.21.15.67:0/1921547842 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6df0019020 con 0x7f6df40fccf0 2023-10-07T23:16:45.533 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:16:45.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6de009b770 msgr2=0x7f6de009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6de009b770 0x7f6de009dc20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6de4007f50 tx=0x7f6de4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:45.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 msgr2=0x7f6df410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 0x7f6df410d960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6df000ebf0 tx=0x7f6df000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:45.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 shutdown_connections 2023-10-07T23:16:45.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df40fe890 0x7f6df4114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f6de009b770 0x7f6de009dc20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df40fccf0 0x7f6df410d960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 --2- 172.21.15.67:0/1921547842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6df40fdf00 0x7f6df410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 >> 172.21.15.67:0/1921547842 conn(0x7f6df40f8450 msgr2=0x7f6df40fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:45.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 shutdown_connections 2023-10-07T23:16:45.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.535+0000 7f6dfab9d700 1 -- 172.21.15.67:0/1921547842 wait complete. 2023-10-07T23:16:45.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:16:45.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 -- 172.21.15.67:0/1652339990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02800fefc0 msgr2=0x7f02800ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 --2- 172.21.15.67:0/1652339990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02800fefc0 0x7f02800ff3d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f027c0049c0 tx=0x7f027c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:45.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 -- 172.21.15.67:0/1652339990 shutdown_connections 2023-10-07T23:16:45.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 --2- 172.21.15.67:0/1652339990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0280100b60 0x7f0280107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 --2- 172.21.15.67:0/1652339990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02800fefc0 0x7f02800ff3d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 --2- 172.21.15.67:0/1652339990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02801001d0 0x7f0280100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 -- 172.21.15.67:0/1652339990 >> 172.21.15.67:0/1652339990 conn(0x7f02800fa630 msgr2=0x7f02800fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:45.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 -- 172.21.15.67:0/1652339990 shutdown_connections 2023-10-07T23:16:45.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.743+0000 7f028679d700 1 -- 172.21.15.67:0/1652339990 wait complete. 2023-10-07T23:16:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.744+0000 7f028679d700 1 Processor -- start 2023-10-07T23:16:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.744+0000 7f028679d700 1 -- start start 2023-10-07T23:16:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.744+0000 7f028679d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02800fefc0 0x7f028010fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028679d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 0x7f02801102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028679d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0280100b60 0x7f0280117310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028679d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280117950 con 0x7f02801001d0 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028679d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280117ac0 con 0x7f0280100b60 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028679d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280117c80 con 0x7f02800fefc0 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f0284f9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 0x7f02801102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02800fefc0 0x7f028010fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f0284f9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 0x7f02801102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44262/0 (socket says 172.21.15.67:44262) 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f0284f9a700 1 -- 172.21.15.67:0/2069389901 learned_addr learned my addr 172.21.15.67:0/2069389901 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f0285f9c700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0280100b60 0x7f0280117310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 -- 172.21.15.67:0/2069389901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0280100b60 msgr2=0x7f0280117310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0280100b60 0x7f0280117310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 -- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 msgr2=0x7f02801102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 0x7f02801102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 -- 172.21.15.67:0/2069389901 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780048f0 con 0x7f02800fefc0 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f0284f9a700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 0x7f02801102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.745+0000 7f028579b700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02800fefc0 0x7f028010fd80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f027c004890 tx=0x7f027c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.746+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00e040 con 0x7f02800fefc0 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.746+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027c013420 con 0x7f02800fefc0 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.746+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c005040 con 0x7f02800fefc0 2023-10-07T23:16:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.746+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c01c660 con 0x7f02800fefc0 2023-10-07T23:16:45.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.746+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280118180 con 0x7f02800fefc0 2023-10-07T23:16:45.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.747+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0280106280 con 0x7f02800fefc0 2023-10-07T23:16:45.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.749+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f027c013590 con 0x7f02800fefc0 2023-10-07T23:16:45.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.750+0000 7f0275ffb700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f026c09b770 0x7f026c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:45.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.750+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f027c0c6d90 con 0x7f02800fefc0 2023-10-07T23:16:45.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.750+0000 7f0284f9a700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f026c09b770 0x7f026c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:45.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.751+0000 7f0284f9a700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f026c09b770 0x7f026c09dc20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f027000e5e0 tx=0x7f027000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:45.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:45.752+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f027c092240 con 0x7f02800fefc0 2023-10-07T23:16:46.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.005+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f028010a7b0 con 0x7f02800fefc0 2023-10-07T23:16:46.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.006+0000 7f0275ffb700 1 -- 172.21.15.67:0/2069389901 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f027c095860 con 0x7f02800fefc0 2023-10-07T23:16:46.007 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:46.007 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:16:46.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f026c09b770 msgr2=0x7f026c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f026c09b770 0x7f026c09dc20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f027000e5e0 tx=0x7f027000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:46.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02800fefc0 msgr2=0x7f028010fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02800fefc0 0x7f028010fd80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f027c004890 tx=0x7f027c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:46.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 shutdown_connections 2023-10-07T23:16:46.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0280100b60 0x7f0280117310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f026c09b770 0x7f026c09dc20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02800fefc0 0x7f028010fd80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 --2- 172.21.15.67:0/2069389901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02801001d0 0x7f02801102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.008+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 >> 172.21.15.67:0/2069389901 conn(0x7f02800fa630 msgr2=0x7f0280103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:46.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.009+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 shutdown_connections 2023-10-07T23:16:46.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.009+0000 7f028679d700 1 -- 172.21.15.67:0/2069389901 wait complete. 2023-10-07T23:16:46.011 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:16:46.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:16:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 -- 172.21.15.67:0/4225512290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60070570 msgr2=0x7efd600709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 --2- 172.21.15.67:0/4225512290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60070570 0x7efd600709c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd50004900 tx=0x7efd50009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 -- 172.21.15.67:0/4225512290 shutdown_connections 2023-10-07T23:16:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 --2- 172.21.15.67:0/4225512290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd60071bc0 0x7efd60070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 --2- 172.21.15.67:0/4225512290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60104b80 0x7efd60107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 --2- 172.21.15.67:0/4225512290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60070570 0x7efd600709c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 -- 172.21.15.67:0/4225512290 >> 172.21.15.67:0/4225512290 conn(0x7efd600f8460 msgr2=0x7efd600fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 -- 172.21.15.67:0/4225512290 shutdown_connections 2023-10-07T23:16:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.217+0000 7efd66f9d700 1 -- 172.21.15.67:0/4225512290 wait complete. 2023-10-07T23:16:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 Processor -- start 2023-10-07T23:16:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 -- start start 2023-10-07T23:16:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd60070570 0x7efd6010fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 0x7efd60110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60104b80 0x7efd60117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd601177a0 con 0x7efd60071bc0 2023-10-07T23:16:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd60117910 con 0x7efd60070570 2023-10-07T23:16:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.218+0000 7efd66f9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd60117ad0 con 0x7efd60104b80 2023-10-07T23:16:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 0x7efd60110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 0x7efd60110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44284/0 (socket says 172.21.15.67:44284) 2023-10-07T23:16:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 -- 172.21.15.67:0/600608450 learned_addr learned my addr 172.21.15.67:0/600608450 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6679c700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60104b80 0x7efd60117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd65f9b700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd60070570 0x7efd6010fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 -- 172.21.15.67:0/600608450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60104b80 msgr2=0x7efd60117190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60104b80 0x7efd60117190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 -- 172.21.15.67:0/600608450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd60070570 msgr2=0x7efd6010fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd60070570 0x7efd6010fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 -- 172.21.15.67:0/600608450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd58009940 con 0x7efd60071bc0 2023-10-07T23:16:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.219+0000 7efd6579a700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 0x7efd60110100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd50009040 tx=0x7efd50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.220+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5000e040 con 0x7efd60071bc0 2023-10-07T23:16:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.220+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5001c430 con 0x7efd60071bc0 2023-10-07T23:16:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.220+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50012650 con 0x7efd60071bc0 2023-10-07T23:16:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.220+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd50005040 con 0x7efd60071bc0 2023-10-07T23:16:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.220+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd60117ff0 con 0x7efd60071bc0 2023-10-07T23:16:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.221+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd601032d0 con 0x7efd60071bc0 2023-10-07T23:16:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.223+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7efd50012a80 con 0x7efd60071bc0 2023-10-07T23:16:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.224+0000 7efd567fc700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd4c09b7f0 0x7efd4c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.224+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7efd500c65d0 con 0x7efd60071bc0 2023-10-07T23:16:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.224+0000 7efd65f9b700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd4c09b7f0 0x7efd4c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.225+0000 7efd65f9b700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd4c09b7f0 0x7efd4c09dca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009b80 tx=0x7efd5c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.228+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7efd50091b30 con 0x7efd60071bc0 2023-10-07T23:16:46.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:46 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/752708895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:16:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:46 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1921547842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:16:46.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:46 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2069389901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:46.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.433+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd6002b450 con 0x7efd60071bc0 2023-10-07T23:16:46.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.434+0000 7efd567fc700 1 -- 172.21.15.67:0/600608450 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7efd5001c5a0 con 0x7efd60071bc0 2023-10-07T23:16:46.435 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:46.435 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:16:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd4c09b7f0 msgr2=0x7efd4c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd4c09b7f0 0x7efd4c09dca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009b80 tx=0x7efd5c00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 msgr2=0x7efd60110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 0x7efd60110100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd50009040 tx=0x7efd50009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 shutdown_connections 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd60070570 0x7efd6010fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd4c09b7f0 0x7efd4c09dca0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd60104b80 0x7efd60117190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 --2- 172.21.15.67:0/600608450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd60071bc0 0x7efd60110100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 >> 172.21.15.67:0/600608450 conn(0x7efd600f8460 msgr2=0x7efd600fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 shutdown_connections 2023-10-07T23:16:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.437+0000 7efd66f9d700 1 -- 172.21.15.67:0/600608450 wait complete. 2023-10-07T23:16:46.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:46 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/752708895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:16:46.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:46 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1921547842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:16:46.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:46 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2069389901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:46.453 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:46 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/752708895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:16:46.453 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:46 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1921547842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:16:46.453 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:46 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2069389901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:46.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:15:25.227955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:15:26.323230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-07T23:15:29.861861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:16:46.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:16:46.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 -- 172.21.15.67:0/3982070878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fccf0 msgr2=0x7fdc440fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3982070878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fccf0 0x7fdc440fd100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc38004900 tx=0x7fdc38009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:46.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 -- 172.21.15.67:0/3982070878 shutdown_connections 2023-10-07T23:16:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3982070878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fccf0 0x7fdc440fd100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3982070878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc440fe890 0x7fdc44105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3982070878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc440fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 -- 172.21.15.67:0/3982070878 >> 172.21.15.67:0/3982070878 conn(0x7fdc440f8450 msgr2=0x7fdc440fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 -- 172.21.15.67:0/3982070878 shutdown_connections 2023-10-07T23:16:46.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.649+0000 7fdc4999d700 1 -- 172.21.15.67:0/3982070878 wait complete. 2023-10-07T23:16:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.650+0000 7fdc4999d700 1 Processor -- start 2023-10-07T23:16:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.650+0000 7fdc4999d700 1 -- start start 2023-10-07T23:16:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4999d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc440fccf0 0x7fdc44071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4999d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc4406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4999d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fe890 0x7fdc440704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4999d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44070c40 con 0x7fdc440fdf00 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4999d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44070db0 con 0x7fdc440fe890 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4999d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44117ad0 con 0x7fdc440fccf0 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc4406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4899b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc440fccf0 0x7fdc44071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc4406ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44290/0 (socket says 172.21.15.67:44290) 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 -- 172.21.15.67:0/3189897115 learned_addr learned my addr 172.21.15.67:0/3189897115 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc4919c700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fe890 0x7fdc440704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 -- 172.21.15.67:0/3189897115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc440fccf0 msgr2=0x7fdc44071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc440fccf0 0x7fdc44071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 -- 172.21.15.67:0/3189897115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fe890 msgr2=0x7fdc440704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fe890 0x7fdc440704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.651+0000 7fdc43fff700 1 -- 172.21.15.67:0/3189897115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009940 con 0x7fdc440fdf00 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.652+0000 7fdc43fff700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc4406ff90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000ebc0 tx=0x7fdc3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.652+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30007cc0 con 0x7fdc440fdf00 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.652+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38005040 con 0x7fdc440fdf00 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.652+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3000f040 con 0x7fdc440fdf00 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.652+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000c840 con 0x7fdc440fdf00 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.652+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc441180d0 con 0x7fdc440fdf00 2023-10-07T23:16:46.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.653+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc44108530 con 0x7fdc440fdf00 2023-10-07T23:16:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.655+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fdc3000cc70 con 0x7fdc440fdf00 2023-10-07T23:16:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.656+0000 7fdc417fa700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdc3409b770 0x7fdc3409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.656+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fdc300c4e10 con 0x7fdc440fdf00 2023-10-07T23:16:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.656+0000 7fdc4899b700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdc3409b770 0x7fdc3409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.657+0000 7fdc4899b700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdc3409b770 0x7fdc3409dc20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc380047d0 tx=0x7fdc38009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.660+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fdc3004da00 con 0x7fdc440fdf00 2023-10-07T23:16:46.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.861+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc44103bb0 con 0x7fdc440fdf00 2023-10-07T23:16:46.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.862+0000 7fdc417fa700 1 -- 172.21.15.67:0/3189897115 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc30019020 con 0x7fdc440fdf00 2023-10-07T23:16:46.863 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdc3409b770 msgr2=0x7fdc3409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdc3409b770 0x7fdc3409dc20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc380047d0 tx=0x7fdc38009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 msgr2=0x7fdc4406ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc4406ff90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000ebc0 tx=0x7fdc3000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 shutdown_connections 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc440fe890 0x7fdc440704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdc3409b770 0x7fdc3409dc20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc440fccf0 0x7fdc44071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 --2- 172.21.15.67:0/3189897115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440fdf00 0x7fdc4406ff90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 >> 172.21.15.67:0/3189897115 conn(0x7fdc440f8450 msgr2=0x7fdc440fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 shutdown_connections 2023-10-07T23:16:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:46.864+0000 7fdc4999d700 1 -- 172.21.15.67:0/3189897115 wait complete. 2023-10-07T23:16:46.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 -- 172.21.15.67:0/2988265569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fdf00 msgr2=0x7fc9200fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 --2- 172.21.15.67:0/2988265569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fdf00 0x7fc9200fe350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc914004900 tx=0x7fc914009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 -- 172.21.15.67:0/2988265569 shutdown_connections 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 --2- 172.21.15.67:0/2988265569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fe890 0x7fc920105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 --2- 172.21.15.67:0/2988265569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9200fccf0 0x7fc9200fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 --2- 172.21.15.67:0/2988265569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fdf00 0x7fc9200fe350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 -- 172.21.15.67:0/2988265569 >> 172.21.15.67:0/2988265569 conn(0x7fc9200f8450 msgr2=0x7fc9200fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 -- 172.21.15.67:0/2988265569 shutdown_connections 2023-10-07T23:16:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.075+0000 7fc92519d700 1 -- 172.21.15.67:0/2988265569 wait complete. 2023-10-07T23:16:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 Processor -- start 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 -- start start 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 0x7fc92010fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9200fdf00 0x7fc9201100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fe890 0x7fc920117180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920117790 con 0x7fc9200fccf0 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920117900 con 0x7fc9200fe890 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92519d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920117ac0 con 0x7fc9200fdf00 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 0x7fc92010fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 0x7fc92010fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44306/0 (socket says 172.21.15.67:44306) 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc91ffff700 1 -- 172.21.15.67:0/1721752177 learned_addr learned my addr 172.21.15.67:0/1721752177 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.076+0000 7fc92499c700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fe890 0x7fc920117180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91f7fe700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9200fdf00 0x7fc9201100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91ffff700 1 -- 172.21.15.67:0/1721752177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9200fdf00 msgr2=0x7fc9201100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91ffff700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9200fdf00 0x7fc9201100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91ffff700 1 -- 172.21.15.67:0/1721752177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fe890 msgr2=0x7fc920117180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91ffff700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fe890 0x7fc920117180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91ffff700 1 -- 172.21.15.67:0/1721752177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc914005040 con 0x7fc9200fccf0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91ffff700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 0x7fc92010fbb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc91000ebc0 tx=0x7fc91000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100047d0 con 0x7fc9200fccf0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc91000f040 con 0x7fc9200fccf0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc920117d40 con 0x7fc9200fccf0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.077+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91000c930 con 0x7fc9200fccf0 2023-10-07T23:16:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.078+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc920118180 con 0x7fc9200fccf0 2023-10-07T23:16:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.079+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920108550 con 0x7fc9200fccf0 2023-10-07T23:16:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.081+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fc9100090d0 con 0x7fc9200fccf0 2023-10-07T23:16:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.082+0000 7fc91cff9700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc90809b770 0x7fc90809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.082+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fc9100c4e30 con 0x7fc9200fccf0 2023-10-07T23:16:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.082+0000 7fc91f7fe700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc90809b770 0x7fc90809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.083+0000 7fc91f7fe700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc90809b770 0x7fc90809dc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc914004f70 tx=0x7fc914009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.086+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fc9100902e0 con 0x7fc9200fccf0 2023-10-07T23:16:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:47 smithi067 ceph-mon[82723]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:47 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/600608450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:47.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:47 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3189897115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:16:47.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.294+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc9201186a0 con 0x7fc9200fccf0 2023-10-07T23:16:47.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.294+0000 7fc91cff9700 1 -- 172.21.15.67:0/1721752177 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc9100190a0 con 0x7fc9200fccf0 2023-10-07T23:16:47.295 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc90809b770 msgr2=0x7fc90809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc90809b770 0x7fc90809dc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc914004f70 tx=0x7fc914009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 msgr2=0x7fc92010fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 0x7fc92010fbb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc91000ebc0 tx=0x7fc91000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 shutdown_connections 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200fe890 0x7fc920117180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc90809b770 0x7fc90809dc20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200fccf0 0x7fc92010fbb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 --2- 172.21.15.67:0/1721752177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9200fdf00 0x7fc9201100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.296+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 >> 172.21.15.67:0/1721752177 conn(0x7fc9200f8450 msgr2=0x7fc920103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.297+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 shutdown_connections 2023-10-07T23:16:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.297+0000 7fc92519d700 1 -- 172.21.15.67:0/1721752177 wait complete. 2023-10-07T23:16:47.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:16:47.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:47 smithi123 ceph-mon[122926]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:47.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:47 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/600608450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:47.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:47 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3189897115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:16:47.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:47 smithi086 ceph-mon[122131]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:47.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:47 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/600608450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:47.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:47 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3189897115' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1636680546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 msgr2=0x7f9fec0fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1636680546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 0x7f9fec0fe500 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00049f0 tx=0x7f9fe0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1636680546 shutdown_connections 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1636680546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fec0fce90 0x7f9fec0fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1636680546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fea40 0x7f9fec105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1636680546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 0x7f9fec0fe500 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1636680546 >> 172.21.15.67:0/1636680546 conn(0x7f9fec0f8660 msgr2=0x7f9fec0faa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1636680546 shutdown_connections 2023-10-07T23:16:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.507+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1636680546 wait complete. 2023-10-07T23:16:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 Processor -- start 2023-10-07T23:16:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 -- start start 2023-10-07T23:16:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 0x7f9fec10fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 0x7f9fec110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fec0fea40 0x7f9fec117270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1178b0 con 0x7f9fec0fe090 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec117a20 con 0x7f9fec0fea40 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.508+0000 7f9ff1fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec117be0 con 0x7f9fec0fce90 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feaffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 0x7f9fec110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 0x7f9fec10fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 0x7f9fec10fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34534/0 (socket says 172.21.15.67:34534) 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 -- 172.21.15.67:0/1625733113 learned_addr learned my addr 172.21.15.67:0/1625733113 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9febfff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fec0fea40 0x7f9fec117270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 -- 172.21.15.67:0/1625733113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fec0fea40 msgr2=0x7f9fec117270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fec0fea40 0x7f9fec117270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 -- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 msgr2=0x7f9fec110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 0x7f9fec110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 -- 172.21.15.67:0/1625733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc005040 con 0x7f9fec0fce90 2023-10-07T23:16:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9feb7fe700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 0x7f9fec10fce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009750 tx=0x7f9fdc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.509+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc007c80 con 0x7f9fec0fce90 2023-10-07T23:16:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.510+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fdc00f040 con 0x7f9fec0fce90 2023-10-07T23:16:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.510+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe0005040 con 0x7f9fec0fce90 2023-10-07T23:16:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.510+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc013c20 con 0x7f9fec0fce90 2023-10-07T23:16:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.510+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1181e0 con 0x7f9fec0fce90 2023-10-07T23:16:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.511+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec108730 con 0x7f9fec0fce90 2023-10-07T23:16:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.513+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9fdc00e7e0 con 0x7f9fec0fce90 2023-10-07T23:16:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.514+0000 7f9fd3fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b8a0 0x7f9fd409dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.514+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f9fdc0c5ca0 con 0x7f9fec0fce90 2023-10-07T23:16:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.514+0000 7f9feaffd700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b8a0 0x7f9fd409dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.515+0000 7f9feaffd700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b8a0 0x7f9fd409dd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004890 tx=0x7f9fe0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.516+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9fdc091150 con 0x7f9fec0fce90 2023-10-07T23:16:47.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.715+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fec118660 con 0x7f9fec0fce90 2023-10-07T23:16:47.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.715+0000 7f9fd3fff700 1 -- 172.21.15.67:0/1625733113 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fdc094770 con 0x7f9fec0fce90 2023-10-07T23:16:47.716 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b8a0 msgr2=0x7f9fd409dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b8a0 0x7f9fd409dd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004890 tx=0x7f9fe0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 msgr2=0x7f9fec10fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 0x7f9fec10fce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009750 tx=0x7f9fdc009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 shutdown_connections 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fec0fea40 0x7f9fec117270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9fd409b8a0 0x7f9fd409dd50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fec0fce90 0x7f9fec10fce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 --2- 172.21.15.67:0/1625733113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fec0fe090 0x7f9fec110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 >> 172.21.15.67:0/1625733113 conn(0x7f9fec0f8660 msgr2=0x7f9fec1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 shutdown_connections 2023-10-07T23:16:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.718+0000 7f9ff1fff700 1 -- 172.21.15.67:0/1625733113 wait complete. 2023-10-07T23:16:47.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-07T23:16:47.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 -- 172.21.15.67:0/188768685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fe8a0 msgr2=0x7f57bc105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/188768685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc105160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f57b4004ca0 tx=0x7f57b4009ca0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 -- 172.21.15.67:0/188768685 shutdown_connections 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/188768685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc105160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/188768685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57bc0fcd00 0x7f57bc0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/188768685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fdf10 0x7f57bc0fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 -- 172.21.15.67:0/188768685 >> 172.21.15.67:0/188768685 conn(0x7f57bc0f8420 msgr2=0x7f57bc0fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 -- 172.21.15.67:0/188768685 shutdown_connections 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.929+0000 7f57c0b9d700 1 -- 172.21.15.67:0/188768685 wait complete. 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.930+0000 7f57c0b9d700 1 Processor -- start 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.930+0000 7f57c0b9d700 1 -- start start 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57c0b9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fcd00 0x7f57bc10f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57c0b9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57bc0fdf10 0x7f57bc10ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57c0b9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc116f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57c0b9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc1175a0 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57c0b9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc117710 con 0x7f57bc0fcd00 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57c0b9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc1178d0 con 0x7f57bc0fdf10 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc116f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc116f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44334/0 (socket says 172.21.15.67:44334) 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 -- 172.21.15.67:0/1914275301 learned_addr learned my addr 172.21.15.67:0/1914275301 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bb7fe700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fcd00 0x7f57bc10f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57baffd700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57bc0fdf10 0x7f57bc10ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 -- 172.21.15.67:0/1914275301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57bc0fdf10 msgr2=0x7f57bc10ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57bc0fdf10 0x7f57bc10ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 -- 172.21.15.67:0/1914275301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fcd00 msgr2=0x7f57bc10f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:47.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fcd00 0x7f57bc10f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:47.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.931+0000 7f57bbfff700 1 -- 172.21.15.67:0/1914275301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0005040 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.932+0000 7f57bbfff700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc116f90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57b4004c70 tx=0x7f57b4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.932+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b400e030 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.932+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b4009550 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.932+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b40048f0 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.932+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b40126a0 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.932+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc117ef0 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.933+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57bc108550 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.935+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f57b4012ad0 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.936+0000 7f57a3fff700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f57a409b7f0 0x7f57a409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:47.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.936+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f57b40c7760 con 0x7f57bc0fe8a0 2023-10-07T23:16:47.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.937+0000 7f57bb7fe700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f57a409b7f0 0x7f57a409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:47.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.937+0000 7f57bb7fe700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f57a409b7f0 0x7f57a409dca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009b80 tx=0x7f57ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:47.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:47.938+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f57b4092c10 con 0x7f57bc0fe8a0 2023-10-07T23:16:48.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.141+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f57bc118080 con 0x7f57bc0fe8a0 2023-10-07T23:16:48.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.142+0000 7f57a3fff700 1 -- 172.21.15.67:0/1914275301 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f57b40170a0 con 0x7f57bc0fe8a0 2023-10-07T23:16:48.142 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f57a409b7f0 msgr2=0x7f57a409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f57a409b7f0 0x7f57a409dca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009b80 tx=0x7f57ac00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 msgr2=0x7f57bc116f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc116f90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57b4004c70 tx=0x7f57b4009ca0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 shutdown_connections 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc0fcd00 0x7f57bc10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f57a409b7f0 0x7f57a409dca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57bc0fe8a0 0x7f57bc116f90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 --2- 172.21.15.67:0/1914275301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57bc0fdf10 0x7f57bc10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 >> 172.21.15.67:0/1914275301 conn(0x7f57bc0f8420 msgr2=0x7f57bc06fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:48.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 shutdown_connections 2023-10-07T23:16:48.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.144+0000 7f57c0b9d700 1 -- 172.21.15.67:0/1914275301 wait complete. 2023-10-07T23:16:48.163 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 -- 172.21.15.67:0/4203334083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fcce0 msgr2=0x7f86480fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 --2- 172.21.15.67:0/4203334083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fcce0 0x7f86480fd0f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f863c0049c0 tx=0x7f863c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 -- 172.21.15.67:0/4203334083 shutdown_connections 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 --2- 172.21.15.67:0/4203334083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86480fe880 0x7f8648105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 --2- 172.21.15.67:0/4203334083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fcce0 0x7f86480fd0f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 --2- 172.21.15.67:0/4203334083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86480fdef0 0x7f86480fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 -- 172.21.15.67:0/4203334083 >> 172.21.15.67:0/4203334083 conn(0x7f86480f8460 msgr2=0x7f86480fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 -- 172.21.15.67:0/4203334083 shutdown_connections 2023-10-07T23:16:48.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.353+0000 7f864d99d700 1 -- 172.21.15.67:0/4203334083 wait complete. 2023-10-07T23:16:48.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 Processor -- start 2023-10-07T23:16:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 -- start start 2023-10-07T23:16:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86480fcce0 0x7f864810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86480fdef0 0x7f864810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 0x7f8648114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648115500 con 0x7f86480fe880 2023-10-07T23:16:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648115670 con 0x7f86480fdef0 2023-10-07T23:16:48.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d99d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648115830 con 0x7f86480fcce0 2023-10-07T23:16:48.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d19c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 0x7f8648114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d19c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 0x7f8648114ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44352/0 (socket says 172.21.15.67:44352) 2023-10-07T23:16:48.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864d19c700 1 -- 172.21.15.67:0/1598800946 learned_addr learned my addr 172.21.15.67:0/1598800946 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:48.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f8647fff700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86480fdef0 0x7f864810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.354+0000 7f864c99b700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86480fcce0 0x7f864810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d19c700 1 -- 172.21.15.67:0/1598800946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86480fcce0 msgr2=0x7f864810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d19c700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86480fcce0 0x7f864810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d19c700 1 -- 172.21.15.67:0/1598800946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86480fdef0 msgr2=0x7f864810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d19c700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86480fdef0 0x7f864810de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d19c700 1 -- 172.21.15.67:0/1598800946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634005040 con 0x7f86480fe880 2023-10-07T23:16:48.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d19c700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 0x7f8648114ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8640001d80 tx=0x7f8640009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400048f0 con 0x7f86480fe880 2023-10-07T23:16:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c005040 con 0x7f86480fe880 2023-10-07T23:16:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864000e8b0 con 0x7f86480fe880 2023-10-07T23:16:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640009650 con 0x7f86480fe880 2023-10-07T23:16:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.355+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8648115e70 con 0x7f86480fe880 2023-10-07T23:16:48.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.356+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86481084d0 con 0x7f86480fe880 2023-10-07T23:16:48.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.358+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f864000ec90 con 0x7f86480fe880 2023-10-07T23:16:48.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.359+0000 7f86457fa700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f863809b770 0x7f863809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.359+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f86400c4fa0 con 0x7f86480fe880 2023-10-07T23:16:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.360+0000 7f864c99b700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f863809b770 0x7f863809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.360+0000 7f864c99b700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f863809b770 0x7f863809dc20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f863c0098c0 tx=0x7f863c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.363+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8640093ee0 con 0x7f86480fe880 2023-10-07T23:16:48.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:48 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1721752177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:16:48.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:48 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1625733113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:16:48.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:48 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1721752177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:16:48.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:48 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1625733113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:16:48.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:48 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1721752177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:16:48.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:48 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1625733113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:16:48.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.611+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8648103b70 con 0x7f86480fe880 2023-10-07T23:16:48.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.612+0000 7f86457fa700 1 -- 172.21.15.67:0/1598800946 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f8640019020 con 0x7f86480fe880 2023-10-07T23:16:48.612 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:48.612 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f863809b770 msgr2=0x7f863809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f863809b770 0x7f863809dc20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f863c0098c0 tx=0x7f863c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 msgr2=0x7f8648114ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 0x7f8648114ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8640001d80 tx=0x7f8640009ee0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 shutdown_connections 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f863809b770 0x7f863809dc20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86480fe880 0x7f8648114ef0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86480fcce0 0x7f864810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 --2- 172.21.15.67:0/1598800946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86480fdef0 0x7f864810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 >> 172.21.15.67:0/1598800946 conn(0x7f86480f8460 msgr2=0x7f86480fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:48.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 shutdown_connections 2023-10-07T23:16:48.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.614+0000 7f864d99d700 1 -- 172.21.15.67:0/1598800946 wait complete. 2023-10-07T23:16:48.616 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:16:48.633 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 -- 172.21.15.67:0/1563634386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fdef0 msgr2=0x7f3b140fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 --2- 172.21.15.67:0/1563634386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fdef0 0x7f3b140fe340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b08004900 tx=0x7f3b08009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 -- 172.21.15.67:0/1563634386 shutdown_connections 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 --2- 172.21.15.67:0/1563634386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b140fe880 0x7f3b14105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 --2- 172.21.15.67:0/1563634386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b140fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 --2- 172.21.15.67:0/1563634386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fdef0 0x7f3b140fe340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 -- 172.21.15.67:0/1563634386 >> 172.21.15.67:0/1563634386 conn(0x7f3b140f8460 msgr2=0x7f3b140fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 -- 172.21.15.67:0/1563634386 shutdown_connections 2023-10-07T23:16:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.822+0000 7f3b1959d700 1 -- 172.21.15.67:0/1563634386 wait complete. 2023-10-07T23:16:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 Processor -- start 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 -- start start 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b14071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b140fdef0 0x7f3b1406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fe880 0x7f3b140704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14070c40 con 0x7f3b140fcce0 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14070db0 con 0x7f3b140fe880 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b1959d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14117a40 con 0x7f3b140fdef0 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b13fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b14071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b18d9c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fe880 0x7f3b140704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b13fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b14071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44364/0 (socket says 172.21.15.67:44364) 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b13fff700 1 -- 172.21.15.67:0/2806494757 learned_addr learned my addr 172.21.15.67:0/2806494757 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.823+0000 7f3b137fe700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b140fdef0 0x7f3b1406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b18d9c700 1 -- 172.21.15.67:0/2806494757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b140fdef0 msgr2=0x7f3b1406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b18d9c700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b140fdef0 0x7f3b1406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b18d9c700 1 -- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 msgr2=0x7f3b14071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b18d9c700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b14071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b18d9c700 1 -- 172.21.15.67:0/2806494757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04005040 con 0x7f3b140fe880 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b13fff700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b14071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b18d9c700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fe880 0x7f3b140704d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00ec20 tx=0x7f3b0c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c007b30 con 0x7f3b140fe880 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b08005040 con 0x7f3b140fe880 2023-10-07T23:16:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b141180a0 con 0x7f3b140fe880 2023-10-07T23:16:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.824+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b0c00f040 con 0x7f3b140fe880 2023-10-07T23:16:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.825+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c00c9f0 con 0x7f3b140fe880 2023-10-07T23:16:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.825+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b14103b00 con 0x7f3b140fe880 2023-10-07T23:16:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.828+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f3b0c004030 con 0x7f3b140fe880 2023-10-07T23:16:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.829+0000 7f3b10ff9700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3afc09b770 0x7f3afc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.829+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f3b0c0c5190 con 0x7f3b140fe880 2023-10-07T23:16:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.829+0000 7f3b13fff700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3afc09b770 0x7f3afc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.830+0000 7f3b13fff700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3afc09b770 0x7f3afc09dc20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b040041d0 tx=0x7f3b0400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:48.831+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f3b0c0906d0 con 0x7f3b140fe880 2023-10-07T23:16:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.041+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b14108550 con 0x7f3b140fe880 2023-10-07T23:16:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.043+0000 7f3b10ff9700 1 -- 172.21.15.67:0/2806494757 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7f3b0c093cf0 con 0x7f3b140fe880 2023-10-07T23:16:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:49.044 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3afc09b770 msgr2=0x7f3afc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3afc09b770 0x7f3afc09dc20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b040041d0 tx=0x7f3b0400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fe880 msgr2=0x7f3b140704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fe880 0x7f3b140704d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00ec20 tx=0x7f3b0c00b040 comp rx=0 tx=0).stop 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 shutdown_connections 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b140fe880 0x7f3b140704d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3afc09b770 0x7f3afc09dc20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b140fcce0 0x7f3b14071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 --2- 172.21.15.67:0/2806494757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b140fdef0 0x7f3b1406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 >> 172.21.15.67:0/2806494757 conn(0x7f3b140f8460 msgr2=0x7f3b140fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 shutdown_connections 2023-10-07T23:16:49.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.045+0000 7f3b1959d700 1 -- 172.21.15.67:0/2806494757 wait complete. 2023-10-07T23:16:49.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:16:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 -- 172.21.15.67:0/1100022171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fdef0 msgr2=0x7fb4d00fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/1100022171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fdef0 0x7fb4d00fe340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40049c0 tx=0x7fb4c4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 -- 172.21.15.67:0/1100022171 shutdown_connections 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/1100022171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fe880 0x7fb4d0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/1100022171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4d00fcce0 0x7fb4d00fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/1100022171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fdef0 0x7fb4d00fe340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 -- 172.21.15.67:0/1100022171 >> 172.21.15.67:0/1100022171 conn(0x7fb4d00f8460 msgr2=0x7fb4d00fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 -- 172.21.15.67:0/1100022171 shutdown_connections 2023-10-07T23:16:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.255+0000 7fb4cf59e700 1 -- 172.21.15.67:0/1100022171 wait complete. 2023-10-07T23:16:49.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 Processor -- start 2023-10-07T23:16:49.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 -- start start 2023-10-07T23:16:49.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 0x7fb4d010fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4d00fdef0 0x7fb4d01100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fe880 0x7fb4d0117150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0117760 con 0x7fb4d00fcce0 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d01178d0 con 0x7fb4d00fe880 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cf59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0117a90 con 0x7fb4d00fdef0 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4ce59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 0x7fb4d010fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4ce59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 0x7fb4d010fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44376/0 (socket says 172.21.15.67:44376) 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4ce59c700 1 -- 172.21.15.67:0/3020442201 learned_addr learned my addr 172.21.15.67:0/3020442201 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4ced9d700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fe880 0x7fb4d0117150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.256+0000 7fb4cdd9b700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4d00fdef0 0x7fb4d01100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4ce59c700 1 -- 172.21.15.67:0/3020442201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4d00fdef0 msgr2=0x7fb4d01100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4ce59c700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4d00fdef0 0x7fb4d01100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4ce59c700 1 -- 172.21.15.67:0/3020442201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fe880 msgr2=0x7fb4d0117150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4ce59c700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fe880 0x7fb4d0117150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4ce59c700 1 -- 172.21.15.67:0/3020442201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4005040 con 0x7fb4d00fcce0 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4ce59c700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 0x7fb4d010fb80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000ebc0 tx=0x7fb4c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0007cc0 con 0x7fb4d00fcce0 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c000f040 con 0x7fb4d00fcce0 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d0117d10 con 0x7fb4d00fcce0 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000c960 con 0x7fb4d00fcce0 2023-10-07T23:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.257+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d01181b0 con 0x7fb4d00fcce0 2023-10-07T23:16:49.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.258+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d0108540 con 0x7fb4d00fcce0 2023-10-07T23:16:49.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.260+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb4c00090d0 con 0x7fb4d00fcce0 2023-10-07T23:16:49.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.261+0000 7fb4beffd700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4b809b770 0x7fb4b809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.262+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb4c00c4e10 con 0x7fb4d00fcce0 2023-10-07T23:16:49.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.262+0000 7fb4cdd9b700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4b809b770 0x7fb4b809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.262+0000 7fb4cdd9b700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4b809b770 0x7fb4b809dc20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4007f50 tx=0x7fb4c4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.266+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fb4c00c8030 con 0x7fb4d00fcce0 2023-10-07T23:16:49.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:49 smithi123 ceph-mon[122926]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:16:49.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:49 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1914275301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:16:49.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:49 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1598800946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:49.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:49 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2806494757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:49 smithi086 ceph-mon[122131]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:16:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:49 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1914275301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:16:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:49 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1598800946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:49.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:49 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2806494757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:49 smithi067 ceph-mon[82723]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:16:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:49 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1914275301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:16:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:49 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1598800946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:49.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:49 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2806494757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:16:49.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.514+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4d0118340 con 0x7fb4d00fcce0 2023-10-07T23:16:49.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.515+0000 7fb4beffd700 1 -- 172.21.15.67:0/3020442201 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7fb4c0019020 con 0x7fb4d00fcce0 2023-10-07T23:16:49.515 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:16:49.515 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4b809b770 msgr2=0x7fb4b809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4b809b770 0x7fb4b809dc20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4007f50 tx=0x7fb4c4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 msgr2=0x7fb4d010fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 0x7fb4d010fb80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000ebc0 tx=0x7fb4c000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 shutdown_connections 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d00fe880 0x7fb4d0117150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb4b809b770 0x7fb4b809dc20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4d00fcce0 0x7fb4d010fb80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 --2- 172.21.15.67:0/3020442201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4d00fdef0 0x7fb4d01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 >> 172.21.15.67:0/3020442201 conn(0x7fb4d00f8460 msgr2=0x7fb4d00fa630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 shutdown_connections 2023-10-07T23:16:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.517+0000 7fb4cf59e700 1 -- 172.21.15.67:0/3020442201 wait complete. 2023-10-07T23:16:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:16:49.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-07T23:15:26.869397+0000', 'modified': '2023-10-07T23:15:55.854427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24538, 'mds_2': 24521, 'mds_3': 24467, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14502': {'gid': 14502, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/3010063667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 3010063667}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 3010063667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/1133310774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 1133310774}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 1133310774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.123:6833/1348503065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1348503065}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1348503065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.67:6837/909177824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6836', 'nonce': 909177824}, {'type': 'v1', 'addr': '172.21.15.67:6837', 'nonce': 909177824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/3322109112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 3322109112}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 3322109112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/4209275840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 4209275840}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 4209275840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/1647886755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1647886755}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1647886755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.86:6837/4272295866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 4272295866}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 4272295866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6839/3252673889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 3252673889}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 3252673889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6839/2147538694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 2147538694}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 2147538694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T23:16:49.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-07T23:16:49.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-10-07T23:16:49.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 -- 172.21.15.67:0/2857680156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fcce0 msgr2=0x7f37300fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 --2- 172.21.15.67:0/2857680156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fcce0 0x7f37300fd0f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37200049c0 tx=0x7f3720009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:49.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 -- 172.21.15.67:0/2857680156 shutdown_connections 2023-10-07T23:16:49.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 --2- 172.21.15.67:0/2857680156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37300fe880 0x7f3730105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 --2- 172.21.15.67:0/2857680156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fcce0 0x7f37300fd0f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 --2- 172.21.15.67:0/2857680156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fdef0 0x7f37300fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 -- 172.21.15.67:0/2857680156 >> 172.21.15.67:0/2857680156 conn(0x7f37300f8460 msgr2=0x7f37300fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 -- 172.21.15.67:0/2857680156 shutdown_connections 2023-10-07T23:16:49.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.726+0000 7f372ffff700 1 -- 172.21.15.67:0/2857680156 wait complete. 2023-10-07T23:16:49.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 Processor -- start 2023-10-07T23:16:49.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 -- start start 2023-10-07T23:16:49.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fcce0 0x7f373010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 0x7f3730110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37300fe880 0x7f37301171b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37301177f0 con 0x7f37300fdef0 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3730117960 con 0x7f37300fcce0 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.727+0000 7f372ffff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3730117b20 con 0x7f37300fe880 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 0x7f3730110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 0x7f3730110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44382/0 (socket says 172.21.15.67:44382) 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 -- 172.21.15.67:0/1536515028 learned_addr learned my addr 172.21.15.67:0/1536515028 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372effd700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fcce0 0x7f373010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372f7fe700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37300fe880 0x7f37301171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 -- 172.21.15.67:0/1536515028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37300fe880 msgr2=0x7f37301171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37300fe880 0x7f37301171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 -- 172.21.15.67:0/1536515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fcce0 msgr2=0x7f373010fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fcce0 0x7f373010fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 -- 172.21.15.67:0/1536515028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720005040 con 0x7f37300fdef0 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372effd700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fcce0 0x7f373010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.728+0000 7f372e7fc700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 0x7f3730110160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f372400ebc0 tx=0x7f372400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.729+0000 7f37177fe700 1 -- 172.21.15.67:0/1536515028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724007cc0 con 0x7f37300fdef0 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.729+0000 7f37177fe700 1 -- 172.21.15.67:0/1536515028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372400f040 con 0x7f37300fdef0 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.729+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3730117da0 con 0x7f37300fdef0 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.729+0000 7f37177fe700 1 -- 172.21.15.67:0/1536515028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37240094a0 con 0x7f37300fdef0 2023-10-07T23:16:49.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.729+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37301182d0 con 0x7f37300fdef0 2023-10-07T23:16:49.733 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.732+0000 7f37177fe700 1 -- 172.21.15.67:0/1536515028 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f372400c600 con 0x7f37300fdef0 2023-10-07T23:16:49.734 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 Processor -- start 2023-10-07T23:16:49.734 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 -- start start 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db11850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37177fe700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f371809b7f0 0x7f371809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 0x55ec8db13140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8da66a70 con 0x55ec8db11440 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8da64270 con 0x55ec8db11d90 2023-10-07T23:16:49.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37441aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8da60060 con 0x55ec8db12cf0 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.734+0000 7f37177fe700 1 -- 172.21.15.67:0/1536515028 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f37240c4d70 con 0x7f37300fdef0 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 0x55ec8db13140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db11850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 0x55ec8db13140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34634/0 (socket says 172.21.15.67:34634) 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 -- 172.21.15.67:0/906319801 learned_addr learned my addr 172.21.15.67:0/906319801 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db11850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44398/0 (socket says 172.21.15.67:44398) 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372f7fe700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f371809b7f0 0x7f371809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372effd700 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 -- 172.21.15.67:0/906319801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 msgr2=0x55ec8db126d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db126d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 -- 172.21.15.67:0/906319801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 msgr2=0x55ec8db11850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db11850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 -- 172.21.15.67:0/906319801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec8d8fa270 con 0x55ec8db12cf0 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372e7fc700 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 0x55ec8db13140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3724000c00 tx=0x7f37240c23d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4476eb90a51622d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.735+0000 7f372f7fe700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f371809b7f0 0x7f371809dca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3728004fd0 tx=0x7f372800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f3714ff9700 1 -- 172.21.15.67:0/906319801 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724093a20 con 0x55ec8db12cf0 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f3714ff9700 1 -- 172.21.15.67:0/906319801 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724093840 con 0x55ec8db12cf0 2023-10-07T23:16:49.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f3714ff9700 1 -- 172.21.15.67:0/906319801 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37240098d0 con 0x55ec8db12cf0 2023-10-07T23:16:49.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 -- 172.21.15.67:0/906319801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 msgr2=0x55ec8db13140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 0x55ec8db13140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3724000c00 tx=0x7f37240c23d0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 -- 172.21.15.67:0/906319801 shutdown_connections 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db12cf0 0x55ec8db13140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db11850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 --2- 172.21.15.67:0/906319801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.736+0000 7f37441aab80 1 -- 172.21.15.67:0/906319801 >> 172.21.15.67:0/906319801 conn(0x55ec8db0c470 msgr2=0x55ec8db0c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.737+0000 7f37441aab80 1 -- 172.21.15.67:0/906319801 shutdown_connections 2023-10-07T23:16:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.737+0000 7f37441aab80 1 -- 172.21.15.67:0/906319801 wait complete. 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.737+0000 7f37441aab80 1 -- start start 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.737+0000 7f37441aab80 1 Processor -- start 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db1b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db12cf0 0x55ec8db22260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8da0b890 con 0x55ec8db12cf0 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8da26ea0 con 0x55ec8db11d90 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8d8e2e30 con 0x55ec8db11440 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f37441aab80 10 client.0 _ll_register_callbacks cb 0x55ec8dad32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34650/0 (socket says 172.21.15.67:34650) 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 -- 172.21.15.67:0/3678019877 learned_addr learned my addr 172.21.15.67:0/3678019877 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372e7fc700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db1b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372f7fe700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db12cf0 0x55ec8db22260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 -- 172.21.15.67:0/3678019877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 msgr2=0x55ec8db1b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db1b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 -- 172.21.15.67:0/3678019877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db12cf0 msgr2=0x55ec8db22260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db12cf0 0x55ec8db22260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372effd700 1 -- 172.21.15.67:0/3678019877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37301182d0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.738+0000 7f372e7fc700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db1b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f372effd700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1acc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37200049c0 tx=0x7f3720009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d310dcc4a3e64ae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37127f4700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37127f4700 1 -- 172.21.15.67:0/3678019877 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372000e070 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec8db04ce0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37127f4700 1 -- 172.21.15.67:0/3678019877 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f3720000bc0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37127f4700 1 -- 172.21.15.67:0/3678019877 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3720009ea0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37441aab80 10 client.25027 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ec8d89cfe0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.739+0000 7f37127f4700 1 -- 172.21.15.67:0/3678019877 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec8db04ce0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37127f4700 1 -- 172.21.15.67:0/3678019877 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f37200296f0 con 0x55ec8db11440 2023-10-07T23:16:49.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37441aab80 10 client.25027 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37441aab80 10 client.25027 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37441aab80 20 client.25027 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37441aab80 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db25340 0x55ec8db2b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37441aab80 4 client.25027 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 --> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec8dae1ab0 con 0x55ec8db25340 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.740+0000 7f372e7fc700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db25340 0x55ec8db2b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.741+0000 7f372e7fc700 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db25340 0x55ec8db2b190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be5b7787549db41e server_cookie=ecf43fd27b46ef8a in_seq=0 out_seq=0 2023-10-07T23:16:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.741+0000 7f37127f4700 10 client.25027 ms_handle_connect on v2:172.21.15.123:6832/1348503065 2023-10-07T23:16:49.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.741+0000 7f37127f4700 1 -- 172.21.15.67:0/3678019877 <== mds.4 v2:172.21.15.123:6832/1348503065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ec8dae1ab0 con 0x55ec8db25340 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37127f4700 10 client.25027 handle_command_reply: tid=0 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 client.25027 shutdown 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 msgr2=0x55ec8db1acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1acc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37200049c0 tx=0x7f3720009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 shutdown_connections 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db25340 0x55ec8db2b190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db12cf0 0x55ec8db22260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1acc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37127f4700 0 client.25027 ms_handle_reset on v2:172.21.15.123:6832/1348503065 2023-10-07T23:16:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 --2- 172.21.15.67:0/3678019877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db11d90 0x55ec8db1b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 >> 172.21.15.67:0/3678019877 conn(0x55ec8db0c470 msgr2=0x55ec8db06f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 shutdown_connections 2023-10-07T23:16:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 1 -- 172.21.15.67:0/3678019877 wait complete. 2023-10-07T23:16:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.742+0000 7f37441aab80 20 client.25027 trim_cache size 0 max 16384 2023-10-07T23:16:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 Processor -- start 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 -- start start 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 0x55ec8db1b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8db0cf50 con 0x55ec8db11d90 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8dae1ab0 con 0x55ec8db12cf0 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f37441aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8da0c2b0 con 0x55ec8db11440 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f372e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 0x55ec8db1b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f372e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 0x55ec8db1b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44410/0 (socket says 172.21.15.67:44410) 2023-10-07T23:16:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f372e7fc700 1 -- 172.21.15.67:0/2200801107 learned_addr learned my addr 172.21.15.67:0/2200801107 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f372f7fe700 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.760+0000 7f372effd700 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372e7fc700 1 -- 172.21.15.67:0/2200801107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 msgr2=0x55ec8db1ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372e7fc700 1 -- 172.21.15.67:0/2200801107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 msgr2=0x55ec8db18a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db18a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372e7fc700 1 -- 172.21.15.67:0/2200801107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec8da2e780 con 0x55ec8db11d90 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372f7fe700 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db18a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 0x55ec8db1b1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f37240c2440 tx=0x7f37240c23d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943563c5297f0983 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f37157fa700 1 -- 172.21.15.67:0/2200801107 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37240945e0 con 0x55ec8db11d90 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f37157fa700 1 -- 172.21.15.67:0/2200801107 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724093ee0 con 0x55ec8db11d90 2023-10-07T23:16:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.761+0000 7f37157fa700 1 -- 172.21.15.67:0/2200801107 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37240943b0 con 0x55ec8db11d90 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 -- 172.21.15.67:0/2200801107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 msgr2=0x55ec8db1b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 0x55ec8db1b1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f37240c2440 tx=0x7f37240c23d0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 -- 172.21.15.67:0/2200801107 shutdown_connections 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db18a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11440 0x55ec8db1ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 --2- 172.21.15.67:0/2200801107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11d90 0x55ec8db1b1a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 -- 172.21.15.67:0/2200801107 >> 172.21.15.67:0/2200801107 conn(0x55ec8db0c470 msgr2=0x55ec8db0c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 -- 172.21.15.67:0/2200801107 shutdown_connections 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 -- 172.21.15.67:0/2200801107 wait complete. 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 -- start start 2023-10-07T23:16:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.762+0000 7f37441aab80 1 Processor -- start 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11d90 0x55ec8db1f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db78150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8db786d0 con 0x55ec8db11440 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8db78840 con 0x55ec8db12cf0 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8db78a00 con 0x55ec8db11d90 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f37441aab80 10 client.0 _ll_register_callbacks cb 0x55ec8dad32c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db1f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44418/0 (socket says 172.21.15.67:44418) 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 -- 172.21.15.67:0/2449555524 learned_addr learned my addr 172.21.15.67:0/2449555524 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372f7fe700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db78150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11d90 0x55ec8db1f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 -- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11d90 msgr2=0x55ec8db1f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11d90 0x55ec8db1f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 -- 172.21.15.67:0/2449555524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 msgr2=0x55ec8db78150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db78150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.763+0000 7f372effd700 1 -- 172.21.15.67:0/2449555524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3728016aa0 con 0x55ec8db11440 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11d90 0x55ec8db1f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f372effd700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db1f340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37200040d0 tx=0x7f3720009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb85df2d35ea9fca server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37127f4700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37127f4700 1 -- 172.21.15.67:0/2449555524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372000e040 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3724093840 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37127f4700 1 -- 172.21.15.67:0/2449555524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f3720029d70 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37127f4700 1 -- 172.21.15.67:0/2449555524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3720009ea0 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37441aab80 10 client.14979 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.764+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ec8d8fa270 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37127f4700 1 -- 172.21.15.67:0/2449555524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724093840 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37127f4700 1 -- 172.21.15.67:0/2449555524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f372000c050 con 0x55ec8db11440 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37441aab80 10 client.14979 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37441aab80 10 client.14979 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37441aab80 20 client.14979 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37441aab80 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db7d330 0x55ec8db79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37441aab80 4 client.14979 mds_command: new command op to 24448 tid=0[{"prefix": "status"}] 2023-10-07T23:16:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 --> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ec8da02e00 con 0x55ec8db7d330 2023-10-07T23:16:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.765+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db7d330 0x55ec8db79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f372e7fc700 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db7d330 0x55ec8db79cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b49c823dcbe62660 server_cookie=8c627323f5630ff3 in_seq=0 out_seq=0 2023-10-07T23:16:49.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37127f4700 10 client.14979 ms_handle_connect on v2:172.21.15.123:6832/1348503065 2023-10-07T23:16:49.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37127f4700 1 -- 172.21.15.67:0/2449555524 <== mds.4 v2:172.21.15.123:6832/1348503065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ec8da02e00 con 0x55ec8db7d330 2023-10-07T23:16:49.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37127f4700 10 client.14979 handle_command_reply: tid=0 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 client.14979 shutdown 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 msgr2=0x55ec8db1f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db1f340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37200040d0 tx=0x7f3720009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 shutdown_connections 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x55ec8db7d330 0x55ec8db79cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec8db12cf0 0x55ec8db78150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ec8db11440 0x55ec8db1f340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37127f4700 0 client.14979 ms_handle_reset on v2:172.21.15.123:6832/1348503065 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 --2- 172.21.15.67:0/2449555524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec8db11d90 0x55ec8db1f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 >> 172.21.15.67:0/2449555524 conn(0x55ec8db0c470 msgr2=0x55ec8db1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 shutdown_connections 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 1 -- 172.21.15.67:0/2449555524 wait complete. 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.766+0000 7f37441aab80 20 client.14979 trim_cache size 0 max 16384 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24448, 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:16:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:16:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 54.915966787999999, 2023-10-07T23:16:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:16:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:16:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:16:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 119.98297997900001 2023-10-07T23:16:49.776 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f371809b7f0 msgr2=0x7f371809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f371809b7f0 0x7f371809dca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3728004fd0 tx=0x7f372800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 msgr2=0x7f3730110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 0x7f3730110160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f372400ebc0 tx=0x7f372400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 shutdown_connections 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37300fcce0 0x7f373010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f371809b7f0 0x7f371809dca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37300fe880 0x7f37301171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 --2- 172.21.15.67:0/1536515028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300fdef0 0x7f3730110160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 >> 172.21.15.67:0/1536515028 conn(0x7f37300f8460 msgr2=0x7f3730103780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 shutdown_connections 2023-10-07T23:16:49.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.767+0000 7f372ffff700 1 -- 172.21.15.67:0/1536515028 wait complete. 2023-10-07T23:16:49.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-07T23:16:49.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 -- 172.21.15.67:0/2559253423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 msgr2=0x7f8d300fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 --2- 172.21.15.67:0/2559253423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d300fd110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d20004900 tx=0x7f8d20009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 -- 172.21.15.67:0/2559253423 shutdown_connections 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 --2- 172.21.15.67:0/2559253423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d300fd110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 --2- 172.21.15.67:0/2559253423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 0x7f8d30105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 --2- 172.21.15.67:0/2559253423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d300fdf10 0x7f8d300fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 -- 172.21.15.67:0/2559253423 >> 172.21.15.67:0/2559253423 conn(0x7f8d300f8420 msgr2=0x7f8d300fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 -- 172.21.15.67:0/2559253423 shutdown_connections 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.977+0000 7f8d3499d700 1 -- 172.21.15.67:0/2559253423 wait complete. 2023-10-07T23:16:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.978+0000 7f8d3499d700 1 Processor -- start 2023-10-07T23:16:49.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.978+0000 7f8d3499d700 1 -- start start 2023-10-07T23:16:49.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d3499d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d3010fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d3499d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d300fdf10 0x7f8d30110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d3499d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 0x7f8d30117120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d3499d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30117730 con 0x7f8d300fe8a0 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d3499d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d301178a0 con 0x7f8d300fcd00 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d3499d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30117a60 con 0x7f8d300fdf10 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 0x7f8d30117120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d3010fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d3010fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:50660/0 (socket says 172.21.15.67:50660) 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3621048594 learned_addr learned my addr 172.21.15.67:0/3621048594 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d300fdf10 0x7f8d30110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3621048594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d300fdf10 msgr2=0x7f8d30110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d300fdf10 0x7f8d30110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 msgr2=0x7f8d30117120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 0x7f8d30117120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3621048594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24005040 con 0x7f8d300fcd00 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.979+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 0x7f8d30117120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.980+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d3010fb50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009fd0 tx=0x7f8d20009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.980+0000 7f8d17fff700 1 -- 172.21.15.67:0/3621048594 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2000e040 con 0x7f8d300fcd00 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.980+0000 7f8d17fff700 1 -- 172.21.15.67:0/3621048594 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d2001c430 con 0x7f8d300fcd00 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.980+0000 7f8d17fff700 1 -- 172.21.15.67:0/3621048594 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20012650 con 0x7f8d300fcd00 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.980+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d20005040 con 0x7f8d300fcd00 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.980+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d30117f80 con 0x7f8d300fcd00 2023-10-07T23:16:49.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.983+0000 7f8d17fff700 1 -- 172.21.15.67:0/3621048594 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f8d2001c5a0 con 0x7f8d300fcd00 2023-10-07T23:16:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.984+0000 7f8d44230b80 1 Processor -- start 2023-10-07T23:16:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.984+0000 7f8d44230b80 1 -- start start 2023-10-07T23:16:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.984+0000 7f8d17fff700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8d1809b770 0x7f8d1809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d17fff700 1 -- 172.21.15.67:0/3621048594 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f8d200c6680 con 0x7f8d300fcd00 2023-10-07T23:16:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8d1809b770 0x7f8d1809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d44230b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc334a70 con 0x55a2fc3e0cf0 2023-10-07T23:16:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d44230b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc332270 con 0x55a2fc3df440 2023-10-07T23:16:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d44230b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc32e060 con 0x55a2fc3dfd90 2023-10-07T23:16:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d2ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d2ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44432/0 (socket says 172.21.15.67:44432) 2023-10-07T23:16:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d2ffff700 1 -- 172.21.15.67:0/1808729736 learned_addr learned my addr 172.21.15.67:0/1808729736 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.985+0000 7f8d2effd700 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/1808729736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 msgr2=0x55a2fc3e06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/1808729736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 msgr2=0x55a2fc3e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/1808729736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2fc1c8270 con 0x55a2fc3df440 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8d1809b770 0x7f8d1809dc20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d2801a500 tx=0x7f8d2800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3df850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009a10 tx=0x7f8d200c3ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987d39eda8d9f0c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d157fa700 1 -- 172.21.15.67:0/1808729736 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200957f0 con 0x55a2fc3df440 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d157fa700 1 -- 172.21.15.67:0/1808729736 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d20091b30 con 0x55a2fc3df440 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.986+0000 7f8d157fa700 1 -- 172.21.15.67:0/1808729736 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200c63e0 con 0x55a2fc3df440 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 -- 172.21.15.67:0/1808729736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 msgr2=0x55a2fc3df850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3df850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009a10 tx=0x7f8d200c3ce0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 -- 172.21.15.67:0/1808729736 shutdown_connections 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3df850 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 --2- 172.21.15.67:0/1808729736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 -- 172.21.15.67:0/1808729736 >> 172.21.15.67:0/1808729736 conn(0x55a2fc3da470 msgr2=0x55a2fc3da860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 -- 172.21.15.67:0/1808729736 shutdown_connections 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 -- 172.21.15.67:0/1808729736 wait complete. 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.987+0000 7f8d44230b80 1 -- start start 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 Processor -- start 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc2d9890 con 0x55a2fc3dfd90 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc2f4ea0 con 0x55a2fc3df440 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc1b0e30 con 0x55a2fc3e0cf0 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d44230b80 10 client.0 _ll_register_callbacks cb 0x55a2fc3a12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d2effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d2f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d2effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44438/0 (socket says 172.21.15.67:44438) 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.988+0000 7f8d2effd700 1 -- 172.21.15.67:0/3454072149 learned_addr learned my addr 172.21.15.67:0/3454072149 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3454072149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3e0cf0 msgr2=0x55a2fc3f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 msgr2=0x55a2fc3e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3454072149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2801ae50 con 0x55a2fc3df440 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3e8ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d20098c00 tx=0x7f8d200c3ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c397ada327bc1ad server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3454072149 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200a0070 con 0x55a2fc3df440 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2fc3d2ce0 con 0x55a2fc3df440 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.989+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3454072149 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f8d200b2030 con 0x55a2fc3df440 2023-10-07T23:16:49.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3454072149 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d200957f0 con 0x55a2fc3df440 2023-10-07T23:16:49.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 10 client.25043 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:49.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f8d30117f80 con 0x55a2fc3df440 2023-10-07T23:16:49.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3454072149 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2fc3d2ce0 con 0x55a2fc3df440 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3454072149 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f8d200a3030 con 0x55a2fc3df440 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 10 client.25043 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 10 client.25043 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 20 client.25043 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc3f36f0 0x55a2fc3f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 4 client.25043 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.990+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 --> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2fc3afab0 con 0x55a2fc3f36f0 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.991+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc3f36f0 0x55a2fc3f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.991+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc3f36f0 0x55a2fc3f89d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1eed734972a2511f server_cookie=888ca3cc707b5639 in_seq=0 out_seq=0 2023-10-07T23:16:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.991+0000 7f8d12ff5700 10 client.25043 ms_handle_connect on v2:172.21.15.67:6836/909177824 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3454072149 <== mds.3 v2:172.21.15.67:6836/909177824 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a2fc3afab0 con 0x55a2fc3f36f0 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d12ff5700 10 client.25043 handle_command_reply: tid=0 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 client.25043 shutdown 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 msgr2=0x55a2fc3e8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3e8ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d20098c00 tx=0x7f8d200c3ce0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 shutdown_connections 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3e8ad0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc3f36f0 0x55a2fc3f89d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d12ff5700 0 client.25043 ms_handle_reset on v2:172.21.15.67:6836/909177824 2023-10-07T23:16:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3454072149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 >> 172.21.15.67:0/3454072149 conn(0x55a2fc3da470 msgr2=0x55a2fc3dbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 shutdown_connections 2023-10-07T23:16:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 1 -- 172.21.15.67:0/3454072149 wait complete. 2023-10-07T23:16:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:49.992+0000 7f8d44230b80 20 client.25043 trim_cache size 0 max 16384 2023-10-07T23:16:50.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 Processor -- start 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 -- start start 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3df440 0x55a2fc3e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc3afab0 con 0x55a2fc3e0cf0 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc1c6030 con 0x55a2fc3dfd90 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d44230b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc2da2b0 con 0x55a2fc3df440 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d2ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d2effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d2ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3db390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44446/0 (socket says 172.21.15.67:44446) 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.010+0000 7f8d2ffff700 1 -- 172.21.15.67:0/2247863112 learned_addr learned my addr 172.21.15.67:0/2247863112 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3df440 0x55a2fc3e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2effd700 1 -- 172.21.15.67:0/2247863112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3df440 msgr2=0x55a2fc3e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2effd700 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3df440 0x55a2fc3e6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2effd700 1 -- 172.21.15.67:0/2247863112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 msgr2=0x55a2fc3db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2effd700 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2effd700 1 -- 172.21.15.67:0/2247863112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2fc16afe0 con 0x55a2fc3dfd90 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d2effd700 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e71c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55a2fc3f5720 tx=0x7f8d2400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad6de1b35c4a3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d15ffb700 1 -- 172.21.15.67:0/2247863112 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2401d070 con 0x55a2fc3dfd90 2023-10-07T23:16:50.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d15ffb700 1 -- 172.21.15.67:0/2247863112 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d2400f040 con 0x55a2fc3dfd90 2023-10-07T23:16:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.011+0000 7f8d15ffb700 1 -- 172.21.15.67:0/2247863112 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e040 con 0x55a2fc3dfd90 2023-10-07T23:16:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 -- 172.21.15.67:0/2247863112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3dfd90 msgr2=0x55a2fc3e71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e71c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55a2fc3f5720 tx=0x7f8d2400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 -- 172.21.15.67:0/2247863112 shutdown_connections 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3df440 0x55a2fc3e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 --2- 172.21.15.67:0/2247863112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3e71c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 -- 172.21.15.67:0/2247863112 >> 172.21.15.67:0/2247863112 conn(0x55a2fc3da470 msgr2=0x55a2fc3e3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 -- 172.21.15.67:0/2247863112 shutdown_connections 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 -- 172.21.15.67:0/2247863112 wait complete. 2023-10-07T23:16:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.012+0000 7f8d44230b80 1 -- start start 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 Processor -- start 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc3ecbe0 con 0x55a2fc3e0cf0 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc3ecd50 con 0x55a2fc3df440 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2fc3ecf10 con 0x55a2fc3dfd90 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d44230b80 10 client.0 _ll_register_callbacks cb 0x55a2fc3a12c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d2f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d2f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:50694/0 (socket says 172.21.15.67:50694) 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3086013127 learned_addr learned my addr 172.21.15.67:0/3086013127 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.013+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3086013127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 msgr2=0x55a2fc3eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 msgr2=0x55a2fc3ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2f7fe700 1 -- 172.21.15.67:0/3086013127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d24009170 con 0x55a2fc3df440 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2ffff700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d2f7fe700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3eb190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8d20095780 tx=0x7f8d200c3ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=addac9a4b6703692 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3086013127 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200a0070 con 0x55a2fc3df440 2023-10-07T23:16:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3086013127 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f8d200b2040 con 0x55a2fc3df440 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d20091b30 con 0x55a2fc3df440 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3086013127 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d2001bee0 con 0x55a2fc3df440 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d44230b80 10 client.25049 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a2fc2fc780 con 0x55a2fc3df440 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.014+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3086013127 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d20091b30 con 0x55a2fc3df440 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3086013127 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f8d2001c880 con 0x55a2fc3df440 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d44230b80 10 client.25049 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d44230b80 10 client.25049 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d44230b80 20 client.25049 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc44a9b0 0x55a2fc447600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d44230b80 4 client.25049 mds_command: new command op to 24467 tid=0[{"prefix": "status"}] 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 --> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2fc2d0e00 con 0x55a2fc44a9b0 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc44a9b0 0x55a2fc447600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d12ff5700 10 client.25049 ms_handle_connect on v2:172.21.15.67:6836/909177824 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.015+0000 7f8d2effd700 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc44a9b0 0x55a2fc447600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7a8bfbb70e72847 server_cookie=fe24079be652ddea in_seq=0 out_seq=0 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d12ff5700 1 -- 172.21.15.67:0/3086013127 <== mds.3 v2:172.21.15.67:6836/909177824 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a2fc2d0e00 con 0x55a2fc44a9b0 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d12ff5700 10 client.25049 handle_command_reply: tid=0 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 client.25049 shutdown 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 msgr2=0x55a2fc3eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3eb190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8d20095780 tx=0x7f8d200c3ce0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 shutdown_connections 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2fc3df440 0x55a2fc3eb190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55a2fc44a9b0 0x55a2fc447600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a2fc3e0cf0 0x55a2fc3ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d12ff5700 0 client.25049 ms_handle_reset on v2:172.21.15.67:6836/909177824 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 --2- 172.21.15.67:0/3086013127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2fc3dfd90 0x55a2fc3eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 >> 172.21.15.67:0/3086013127 conn(0x55a2fc3da470 msgr2=0x55a2fc3e3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 shutdown_connections 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 1 -- 172.21.15.67:0/3086013127 wait complete. 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.016+0000 7f8d44230b80 20 client.25049 trim_cache size 0 max 16384 2023-10-07T23:16:50.023 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24467, 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 56.175773519000003, 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:16:50.024 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 111.401486248 2023-10-07T23:16:50.025 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:16:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8d1809b770 msgr2=0x7f8d1809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8d1809b770 0x7f8d1809dc20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d2801a500 tx=0x7f8d2800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 msgr2=0x7f8d3010fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d3010fb50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009fd0 tx=0x7f8d20009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 shutdown_connections 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d300fcd00 0x7f8d3010fb50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8d1809b770 0x7f8d1809dc20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d300fe8a0 0x7f8d30117120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 --2- 172.21.15.67:0/3621048594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d300fdf10 0x7f8d30110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 >> 172.21.15.67:0/3621048594 conn(0x7f8d300f8420 msgr2=0x7f8d30103700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 shutdown_connections 2023-10-07T23:16:50.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.017+0000 7f8d3499d700 1 -- 172.21.15.67:0/3621048594 wait complete. 2023-10-07T23:16:50.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-07T23:16:50.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:50 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3020442201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.225+0000 7fb02bfff700 1 -- 172.21.15.67:0/3036449150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c0fefc0 msgr2=0x7fb02c0ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.225+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3036449150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c0fefc0 0x7fb02c0ff3d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0049c0 tx=0x7fb01c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 -- 172.21.15.67:0/3036449150 shutdown_connections 2023-10-07T23:16:50.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3036449150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c100b60 0x7fb02c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3036449150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c0fefc0 0x7fb02c0ff3d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3036449150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb02c1001d0 0x7fb02c100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 -- 172.21.15.67:0/3036449150 >> 172.21.15.67:0/3036449150 conn(0x7fb02c0fa630 msgr2=0x7fb02c0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 -- 172.21.15.67:0/3036449150 shutdown_connections 2023-10-07T23:16:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 -- 172.21.15.67:0/3036449150 wait complete. 2023-10-07T23:16:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.226+0000 7fb02bfff700 1 Processor -- start 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 -- start start 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 0x7fb02c10fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb02c1001d0 0x7fb02c110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c100b60 0x7fb02c1171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c1177b0 con 0x7fb02c100b60 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c117920 con 0x7fb02c1001d0 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c117ae0 con 0x7fb02c0fefc0 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 0x7fb02c10fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c100b60 0x7fb02c1171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb02c1001d0 0x7fb02c110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 0x7fb02c10fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34712/0 (socket says 172.21.15.67:34712) 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.227+0000 7fb02affd700 1 -- 172.21.15.67:0/3132445852 learned_addr learned my addr 172.21.15.67:0/3132445852 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02affd700 1 -- 172.21.15.67:0/3132445852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb02c1001d0 msgr2=0x7fb02c110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02affd700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb02c1001d0 0x7fb02c110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02affd700 1 -- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c100b60 msgr2=0x7fb02c1171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02affd700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c100b60 0x7fb02c1171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02affd700 1 -- 172.21.15.67:0/3132445852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c005040 con 0x7fb02c0fefc0 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02b7fe700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c100b60 0x7fb02c1171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02affd700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 0x7fb02c10fbd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009180 tx=0x7fb01c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb0137fe700 1 -- 172.21.15.67:0/3132445852 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c00e070 con 0x7fb02c0fefc0 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb0137fe700 1 -- 172.21.15.67:0/3132445852 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01c009560 con 0x7fb02c0fefc0 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c117d60 con 0x7fb02c0fefc0 2023-10-07T23:16:50.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb0137fe700 1 -- 172.21.15.67:0/3132445852 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c012630 con 0x7fb02c0fefc0 2023-10-07T23:16:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.228+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c118160 con 0x7fb02c0fefc0 2023-10-07T23:16:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.231+0000 7fb0137fe700 1 -- 172.21.15.67:0/3132445852 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb01c012a60 con 0x7fb02c0fefc0 2023-10-07T23:16:50.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.232+0000 7fb03fbc2b80 1 Processor -- start 2023-10-07T23:16:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 -- start start 2023-10-07T23:16:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2c440 0x561891a2c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2dcf0 0x561891a2e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891981a70 con 0x561891a2cd90 2023-10-07T23:16:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189197f270 con 0x561891a2dcf0 2023-10-07T23:16:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189197b060 con 0x561891a2c440 2023-10-07T23:16:50.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb0137fe700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb01409b770 0x7fb01409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb0137fe700 1 -- 172.21.15.67:0/3132445852 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb01c0c68e0 con 0x7fb02c0fefc0 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb02affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb02affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a2d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44488/0 (socket says 172.21.15.67:44488) 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2dcf0 0x561891a2e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb02affd700 1 -- 172.21.15.67:0/4168093500 learned_addr learned my addr 172.21.15.67:0/4168093500 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb02b7fe700 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2c440 0x561891a2c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.233+0000 7fb02b7fe700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb01409b770 0x7fb01409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02affd700 1 -- 172.21.15.67:0/4168093500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2c440 msgr2=0x561891a2c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02affd700 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2c440 0x561891a2c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02affd700 1 -- 172.21.15.67:0/4168093500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2dcf0 msgr2=0x561891a2e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02affd700 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2dcf0 0x561891a2e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02affd700 1 -- 172.21.15.67:0/4168093500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561891815270 con 0x561891a2cd90 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2dcf0 0x561891a2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02affd700 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a2d6d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0924c0 tx=0x7fb01c0c3f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c695daa2176aba server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb02b7fe700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb01409b770 0x7fb01409dc20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb024004fd0 tx=0x7fb02400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb010ff9700 1 -- 172.21.15.67:0/4168093500 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c096150 con 0x561891a2cd90 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb010ff9700 1 -- 172.21.15.67:0/4168093500 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01c023310 con 0x561891a2cd90 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.234+0000 7fb010ff9700 1 -- 172.21.15.67:0/4168093500 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c09cb10 con 0x561891a2cd90 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4168093500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 msgr2=0x561891a2d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a2d6d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0924c0 tx=0x7fb01c0c3f40 comp rx=0 tx=0).stop 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4168093500 shutdown_connections 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2dcf0 0x561891a2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2c440 0x561891a2c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4168093500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a2d6d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4168093500 >> 172.21.15.67:0/4168093500 conn(0x561891a27470 msgr2=0x561891a27860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4168093500 shutdown_connections 2023-10-07T23:16:50.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.235+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4168093500 wait complete. 2023-10-07T23:16:50.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 -- start start 2023-10-07T23:16:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 Processor -- start 2023-10-07T23:16:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a35b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a36050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891926890 con 0x561891a2cd90 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891941ea0 con 0x561891a2c440 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618917fde30 con 0x561891a2dcf0 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb03fbc2b80 10 client.0 _ll_register_callbacks cb 0x5618919ee2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.236+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a36050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a36050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44490/0 (socket says 172.21.15.67:44490) 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 -- 172.21.15.67:0/4260827555 learned_addr learned my addr 172.21.15.67:0/4260827555 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a35b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 -- 172.21.15.67:0/4260827555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 msgr2=0x561891a3d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a3d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 -- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 msgr2=0x561891a35b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a35b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 -- 172.21.15.67:0/4260827555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb020005040 con 0x561891a2cd90 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a36050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb020014dd0 tx=0x7fb020002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0c898b9c058597 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb00e7f4700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4260827555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000e070 con 0x561891a2cd90 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561891a1fce0 con 0x561891a2cd90 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.237+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4260827555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb020012b20 con 0x561891a2cd90 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4260827555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb020004e90 con 0x561891a2cd90 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 10 client.15009 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fb02c118160 con 0x561891a2cd90 2023-10-07T23:16:50.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4260827555 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561891a1fce0 con 0x561891a2cd90 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4260827555 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7fb020016400 con 0x561891a2cd90 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 10 client.15009 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 10 client.15009 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 20 client.15009 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a40150 0x561891a45f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 4 client.15009 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.238+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 --> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618919fcab0 con 0x561891a40150 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.239+0000 7fb02affd700 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a40150 0x561891a45f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.239+0000 7fb02affd700 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a40150 0x561891a45f70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e54cfc6cad79759 server_cookie=48555546134ec907 in_seq=0 out_seq=0 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.239+0000 7fb00e7f4700 10 client.15009 ms_handle_connect on v2:172.21.15.86:6836/4272295866 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4260827555 <== mds.2 v2:172.21.15.86:6836/4272295866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5618919fcab0 con 0x561891a40150 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb00e7f4700 10 client.15009 handle_command_reply: tid=0 2023-10-07T23:16:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 client.15009 shutdown 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 msgr2=0x561891a36050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a36050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb020014dd0 tx=0x7fb020002040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 shutdown_connections 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a40150 0x561891a45f70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a35b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb00e7f4700 0 client.15009 ms_handle_reset on v2:172.21.15.86:6836/4272295866 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4260827555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a36050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 >> 172.21.15.67:0/4260827555 conn(0x561891a27470 msgr2=0x561891a21f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 shutdown_connections 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4260827555 wait complete. 2023-10-07T23:16:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.240+0000 7fb03fbc2b80 20 client.15009 trim_cache size 0 max 16384 2023-10-07T23:16:50.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 Processor -- start 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 -- start start 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a29140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a30900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a30e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891981a70 con 0x561891a2cd90 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891a27f50 con 0x561891a2c440 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618919fcab0 con 0x561891a2dcf0 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a30900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a30900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44502/0 (socket says 172.21.15.67:44502) 2023-10-07T23:16:50.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.258+0000 7fb02a7fc700 1 -- 172.21.15.67:0/91744172 learned_addr learned my addr 172.21.15.67:0/91744172 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02affd700 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a29140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02b7fe700 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a30e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02a7fc700 1 -- 172.21.15.67:0/91744172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 msgr2=0x561891a30e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a30e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02a7fc700 1 -- 172.21.15.67:0/91744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 msgr2=0x561891a29140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a29140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02a7fc700 1 -- 172.21.15.67:0/91744172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618917b7fe0 con 0x561891a2cd90 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02affd700 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a29140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a30900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb020009b80 tx=0x7fb020002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a60d2908bbf28c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb0117fa700 1 -- 172.21.15.67:0/91744172 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000e070 con 0x561891a2cd90 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb0117fa700 1 -- 172.21.15.67:0/91744172 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb02000f040 con 0x561891a2cd90 2023-10-07T23:16:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.259+0000 7fb0117fa700 1 -- 172.21.15.67:0/91744172 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020000cd0 con 0x561891a2cd90 2023-10-07T23:16:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/91744172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 msgr2=0x561891a30900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a30900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb020009b80 tx=0x7fb020002040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/91744172 shutdown_connections 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2c440 0x561891a29140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a30e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/91744172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2cd90 0x561891a30900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/91744172 >> 172.21.15.67:0/91744172 conn(0x561891a27470 msgr2=0x561891a27840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/91744172 shutdown_connections 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/91744172 wait complete. 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.260+0000 7fb03fbc2b80 1 -- start start 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 Processor -- start 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 0x561891a3a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2cd90 0x561891a3a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891a936c0 con 0x561891a2c440 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891a93830 con 0x561891a2cd90 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561891a939f0 con 0x561891a2dcf0 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb03fbc2b80 10 client.0 _ll_register_callbacks cb 0x5618919ee2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb02affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 0x561891a3a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb02affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 0x561891a3a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44510/0 (socket says 172.21.15.67:44510) 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb02affd700 1 -- 172.21.15.67:0/4054387337 learned_addr learned my addr 172.21.15.67:0/4054387337 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.261+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2cd90 0x561891a3a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02b7fe700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02affd700 1 -- 172.21.15.67:0/4054387337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 msgr2=0x561891a93140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02affd700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a93140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02affd700 1 -- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2cd90 msgr2=0x561891a3a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02affd700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2cd90 0x561891a3a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02affd700 1 -- 172.21.15.67:0/4054387337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c118160 con 0x561891a2c440 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2cd90 0x561891a3a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb02affd700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 0x561891a3a3a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb01c095750 tx=0x7fb01c0c6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b407acd8f7adc795 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb00e7f4700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4054387337 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0096d0 con 0x561891a2c440 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4054387337 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb01c095a50 con 0x561891a2c440 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb01c023310 con 0x561891a2c440 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4054387337 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb01c09ca80 con 0x561891a2c440 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb03fbc2b80 10 client.15015 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.262+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561891949780 con 0x561891a2c440 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4054387337 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01c023310 con 0x561891a2c440 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4054387337 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7fb01c09c710 con 0x561891a2c440 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb03fbc2b80 10 client.15015 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb03fbc2b80 10 client.15015 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb03fbc2b80 20 client.15015 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a98210 0x561891a94be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb03fbc2b80 4 client.15015 mds_command: new command op to 24521 tid=0[{"prefix": "status"}] 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 --> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56189191de00 con 0x561891a98210 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.263+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a98210 0x561891a94be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb02a7fc700 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a98210 0x561891a94be0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49e91eab2815b0d7 server_cookie=12c175aaeb22d952 in_seq=0 out_seq=0 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb00e7f4700 10 client.15015 ms_handle_connect on v2:172.21.15.86:6836/4272295866 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb00e7f4700 1 -- 172.21.15.67:0/4054387337 <== mds.2 v2:172.21.15.86:6836/4272295866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56189191de00 con 0x561891a98210 2023-10-07T23:16:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb00e7f4700 10 client.15015 handle_command_reply: tid=0 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 client.15015 shutdown 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 msgr2=0x561891a3a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 0x561891a3a3a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb01c095750 tx=0x7fb01c0c6040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 shutdown_connections 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x561891a98210 0x561891a94be0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561891a2dcf0 0x561891a93140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561891a2c440 0x561891a3a3a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb00e7f4700 0 client.15015 ms_handle_reset on v2:172.21.15.86:6836/4272295866 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 --2- 172.21.15.67:0/4054387337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561891a2cd90 0x561891a3a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 >> 172.21.15.67:0/4054387337 conn(0x561891a27470 msgr2=0x561891a30900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 shutdown_connections 2023-10-07T23:16:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 1 -- 172.21.15.67:0/4054387337 wait complete. 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.264+0000 7fb03fbc2b80 20 client.15015 trim_cache size 0 max 16384 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24521, 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:16:50.271 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 57.252789217999997, 2023-10-07T23:16:50.272 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:16:50.272 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:16:50.272 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:16:50.272 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 101.19901290999999 2023-10-07T23:16:50.272 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb01409b770 msgr2=0x7fb01409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb01409b770 0x7fb01409dc20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb024004fd0 tx=0x7fb02400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 msgr2=0x7fb02c10fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 0x7fb02c10fbd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009180 tx=0x7fb01c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 shutdown_connections 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb01409b770 0x7fb01409dc20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb02c100b60 0x7fb02c1171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb02c0fefc0 0x7fb02c10fbd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 --2- 172.21.15.67:0/3132445852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb02c1001d0 0x7fb02c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 >> 172.21.15.67:0/3132445852 conn(0x7fb02c0fa630 msgr2=0x7fb02c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 shutdown_connections 2023-10-07T23:16:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.265+0000 7fb02bfff700 1 -- 172.21.15.67:0/3132445852 wait complete. 2023-10-07T23:16:50.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-07T23:16:50.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:50 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3020442201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:50.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:50 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3020442201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 -- 172.21.15.67:0/2397690156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fdef0 msgr2=0x7f23e80fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 --2- 172.21.15.67:0/2397690156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fdef0 0x7f23e80fe340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0049c0 tx=0x7f23dc009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 -- 172.21.15.67:0/2397690156 shutdown_connections 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 --2- 172.21.15.67:0/2397690156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23e80fcce0 0x7f23e80fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 --2- 172.21.15.67:0/2397690156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23e80fe880 0x7f23e8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 --2- 172.21.15.67:0/2397690156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fdef0 0x7f23e80fe340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 -- 172.21.15.67:0/2397690156 >> 172.21.15.67:0/2397690156 conn(0x7f23e80f8460 msgr2=0x7f23e80fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 -- 172.21.15.67:0/2397690156 shutdown_connections 2023-10-07T23:16:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.472+0000 7f23e759e700 1 -- 172.21.15.67:0/2397690156 wait complete. 2023-10-07T23:16:50.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 Processor -- start 2023-10-07T23:16:50.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 -- start start 2023-10-07T23:16:50.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23e80fcce0 0x7f23e810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23e80fdef0 0x7f23e810df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 0x7f23e8114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81155c0 con 0x7f23e80fe880 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8115730 con 0x7f23e80fcce0 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.473+0000 7f23e759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81158f0 con 0x7f23e80fdef0 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 0x7f23e8114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 0x7f23e8114fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44528/0 (socket says 172.21.15.67:44528) 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3299782437 learned_addr learned my addr 172.21.15.67:0/3299782437 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23e80fdef0 0x7f23e810df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e659c700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23e80fcce0 0x7f23e810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3299782437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23e80fdef0 msgr2=0x7f23e810df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23e80fdef0 0x7f23e810df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3299782437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23e80fcce0 msgr2=0x7f23e810d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23e80fcce0 0x7f23e810d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3299782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc005040 con 0x7f23e80fe880 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 0x7f23e8114fb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23e000ec30 tx=0x7f23e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.474+0000 7f23d6ffd700 1 -- 172.21.15.67:0/3299782437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0007b30 con 0x7f23e80fe880 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.475+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e8115bd0 con 0x7f23e80fe880 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.475+0000 7f23d6ffd700 1 -- 172.21.15.67:0/3299782437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23e000f040 con 0x7f23e80fe880 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.475+0000 7f23d6ffd700 1 -- 172.21.15.67:0/3299782437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e000ca30 con 0x7f23e80fe880 2023-10-07T23:16:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.475+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e8116110 con 0x7f23e80fe880 2023-10-07T23:16:50.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.478+0000 7f23d6ffd700 1 -- 172.21.15.67:0/3299782437 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f23e001d400 con 0x7f23e80fe880 2023-10-07T23:16:50.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 Processor -- start 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 -- start start 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23d6ffd700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f23d009b770 0x7f23d009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23d6ffd700 1 -- 172.21.15.67:0/3299782437 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f23e00c5710 con 0x7f23e80fe880 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0d90 0x55f5c73c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c73c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c7315a70 con 0x55f5c73c0d90 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c7313270 con 0x55f5c73c1cf0 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c730f060 con 0x55f5c73c0440 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f23d009b770 0x7f23d009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23e5d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0d90 0x55f5c73c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23e5d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0d90 0x55f5c73c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44534/0 (socket says 172.21.15.67:44534) 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23e659c700 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.479+0000 7f23e5d9b700 1 -- 172.21.15.67:0/2328543940 learned_addr learned my addr 172.21.15.67:0/2328543940 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c73c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 -- 172.21.15.67:0/2328543940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 msgr2=0x55f5c73c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 -- 172.21.15.67:0/2328543940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0d90 msgr2=0x55f5c73c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0d90 0x55f5c73c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 -- 172.21.15.67:0/2328543940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c71a9270 con 0x55f5c73c1cf0 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e659c700 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f23d009b770 0x7f23d009dc20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000e50 tx=0x7f23dc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c73c2140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23e00c2ea0 tx=0x7f23e00c2d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f5c769a416c6e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23cffff700 1 -- 172.21.15.67:0/2328543940 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e000ce60 con 0x55f5c73c1cf0 2023-10-07T23:16:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23cffff700 1 -- 172.21.15.67:0/2328543940 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23e0094f80 con 0x55f5c73c1cf0 2023-10-07T23:16:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.480+0000 7f23cffff700 1 -- 172.21.15.67:0/2328543940 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00c5470 con 0x55f5c73c1cf0 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2328543940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 msgr2=0x55f5c73c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c73c2140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23e00c2ea0 tx=0x7f23e00c2d70 comp rx=0 tx=0).stop 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2328543940 shutdown_connections 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c73c2140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2328543940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0d90 0x55f5c73c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2328543940 >> 172.21.15.67:0/2328543940 conn(0x55f5c73bb470 msgr2=0x55f5c73bb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2328543940 shutdown_connections 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.481+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2328543940 wait complete. 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 -- start start 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 Processor -- start 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c72ba890 con 0x55f5c73c1cf0 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c72d5ea0 con 0x55f5c73c0d90 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c7191e30 con 0x55f5c73c0440 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.482+0000 7f23fb0f4b80 10 client.0 _ll_register_callbacks cb 0x55f5c73822c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44548/0 (socket says 172.21.15.67:44548) 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3639796429 learned_addr learned my addr 172.21.15.67:0/3639796429 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 msgr2=0x55f5c73c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3639796429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 msgr2=0x55f5c73ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 -- 172.21.15.67:0/3639796429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d8005040 con 0x55f5c73c1cf0 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73d1400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23e00912f0 tx=0x7f23e0097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac7c44d7735d7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23cd7fa700 1 -- 172.21.15.67:0/3639796429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e000ce60 con 0x55f5c73c1cf0 2023-10-07T23:16:50.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23cd7fa700 1 -- 172.21.15.67:0/3639796429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f23e00abe40 con 0x55f5c73c1cf0 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.483+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5c73b3ce0 con 0x55f5c73c1cf0 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23cd7fa700 1 -- 172.21.15.67:0/3639796429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23e00941e0 con 0x55f5c73c1cf0 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 10 client.15027 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f23e8116110 con 0x55f5c73c1cf0 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23cd7fa700 1 -- 172.21.15.67:0/3639796429 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5c73b3ce0 con 0x55f5c73c1cf0 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23cd7fa700 1 -- 172.21.15.67:0/3639796429 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f23e0093d70 con 0x55f5c73c1cf0 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 10 client.15027 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 20 client.15027 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c73d4c20 0x55f5c73d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 4 client.15027 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:16:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.484+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 --> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5c7390ab0 con 0x55f5c73d4c20 2023-10-07T23:16:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.485+0000 7f23e659c700 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c73d4c20 0x55f5c73d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.485+0000 7f23e659c700 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c73d4c20 0x55f5c73d9ee0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=353b7f3fd5514267 server_cookie=270bec7f6378050f in_seq=0 out_seq=0 2023-10-07T23:16:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.485+0000 7f23cd7fa700 10 client.15027 ms_handle_connect on v2:172.21.15.123:6838/3252673889 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23cd7fa700 1 -- 172.21.15.67:0/3639796429 <== mds.1 v2:172.21.15.123:6838/3252673889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f5c7390ab0 con 0x55f5c73d4c20 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23cd7fa700 10 client.15027 handle_command_reply: tid=0 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 client.15027 shutdown 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 msgr2=0x55f5c73d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73d1400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23e00912f0 tx=0x7f23e0097040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 shutdown_connections 2023-10-07T23:16:50.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c73d4c20 0x55f5c73d9ee0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73d1400 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23cd7fa700 0 client.15027 ms_handle_reset on v2:172.21.15.123:6838/3252673889 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/3639796429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 >> 172.21.15.67:0/3639796429 conn(0x55f5c73bb470 msgr2=0x55f5c73b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 shutdown_connections 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.486+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/3639796429 wait complete. 2023-10-07T23:16:50.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.487+0000 7f23fb0f4b80 20 client.15027 trim_cache size 0 max 16384 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 Processor -- start 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 -- start start 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c73b7a50 con 0x55f5c73c1cf0 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c7390ae0 con 0x55f5c73c0d90 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c72dad60 con 0x55f5c73c0440 2023-10-07T23:16:50.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.504+0000 7f23e6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e5d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e5d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:50802/0 (socket says 172.21.15.67:50802) 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44550/0 (socket says 172.21.15.67:44550) 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e5d9b700 1 -- 172.21.15.67:0/2463631561 learned_addr learned my addr 172.21.15.67:0/2463631561 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 -- 172.21.15.67:0/2463631561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 msgr2=0x55f5c73c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 -- 172.21.15.67:0/2463631561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 msgr2=0x55f5c73b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 -- 172.21.15.67:0/2463631561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c714bfe0 con 0x55f5c73c1cf0 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e659c700 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73b5bb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23e0090d80 tx=0x7f23e0097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59160bb3085d5cb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23d4ff9700 1 -- 172.21.15.67:0/2463631561 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00941e0 con 0x55f5c73c1cf0 2023-10-07T23:16:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23d4ff9700 1 -- 172.21.15.67:0/2463631561 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23e000ce60 con 0x55f5c73c1cf0 2023-10-07T23:16:50.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.505+0000 7f23d4ff9700 1 -- 172.21.15.67:0/2463631561 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0094880 con 0x55f5c73c1cf0 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2463631561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 msgr2=0x55f5c73b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73b5bb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f23e0090d80 tx=0x7f23e0097040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2463631561 shutdown_connections 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c1cf0 0x55f5c73b5bb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0440 0x55f5c73c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/2463631561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c0d90 0x55f5c73b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2463631561 >> 172.21.15.67:0/2463631561 conn(0x55f5c73bb470 msgr2=0x55f5c73c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2463631561 shutdown_connections 2023-10-07T23:16:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.506+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/2463631561 wait complete. 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 -- start start 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 Processor -- start 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 0x55f5c73c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0d90 0x55f5c7428f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c742b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c742ba50 con 0x55f5c73c0440 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c742bbc0 con 0x55f5c73c1cf0 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c742bd80 con 0x55f5c73c0d90 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23fb0f4b80 10 client.0 _ll_register_callbacks cb 0x55f5c73822c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23e659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 0x55f5c73c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23e659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 0x55f5c73c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44562/0 (socket says 172.21.15.67:44562) 2023-10-07T23:16:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.507+0000 7f23e659c700 1 -- 172.21.15.67:0/182917984 learned_addr learned my addr 172.21.15.67:0/182917984 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e6d9d700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c742b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0d90 0x55f5c7428f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e659c700 1 -- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0d90 msgr2=0x55f5c7428f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e659c700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0d90 0x55f5c7428f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e659c700 1 -- 172.21.15.67:0/182917984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 msgr2=0x55f5c742b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e659c700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c742b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e659c700 1 -- 172.21.15.67:0/182917984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d80155d0 con 0x55f5c73c0440 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0d90 0x55f5c7428f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23e659c700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 0x55f5c73c7290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000c00 tx=0x7f23d8019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e252f9afe45dea server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23cd7fa700 1 -- 172.21.15.67:0/182917984 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8024070 con 0x55f5c73c0440 2023-10-07T23:16:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23e0094f80 con 0x55f5c73c0440 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.508+0000 7f23cd7fa700 1 -- 172.21.15.67:0/182917984 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f23d8007cc0 con 0x55f5c73c0440 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23cd7fa700 1 -- 172.21.15.67:0/182917984 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d801c040 con 0x55f5c73c0440 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 10 client.15033 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f23dc017040 con 0x55f5c73c0440 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23cd7fa700 1 -- 172.21.15.67:0/182917984 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23e0094f80 con 0x55f5c73c0440 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23cd7fa700 1 -- 172.21.15.67:0/182917984 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f23d8027030 con 0x55f5c73c0440 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 10 client.15033 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 10 client.15033 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 20 client.15033 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c7430990 0x55f5c742d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 4 client.15033 mds_command: new command op to 24538 tid=0[{"prefix": "status"}] 2023-10-07T23:16:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 --> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f5c72b1e00 con 0x55f5c7430990 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.509+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c7430990 0x55f5c742d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23e5d9b700 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c7430990 0x55f5c742d300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f7cf07aa92ded83 server_cookie=c90271a05add16ec in_seq=0 out_seq=0 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23cd7fa700 10 client.15033 ms_handle_connect on v2:172.21.15.123:6838/3252673889 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23cd7fa700 1 -- 172.21.15.67:0/182917984 <== mds.1 v2:172.21.15.123:6838/3252673889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f5c72b1e00 con 0x55f5c7430990 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23cd7fa700 10 client.15033 handle_command_reply: tid=0 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23fb0f4b80 1 client.15033 shutdown 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 msgr2=0x55f5c73c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 0x55f5c73c7290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000c00 tx=0x7f23d8019040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.510+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 shutdown_connections 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x55f5c7430990 0x55f5c742d300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c73c1cf0 0x55f5c742b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5c73c0440 0x55f5c73c7290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 --2- 172.21.15.67:0/182917984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f5c73c0d90 0x55f5c7428f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 >> 172.21.15.67:0/182917984 conn(0x55f5c73bb470 msgr2=0x55f5c73b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23cd7fa700 0 client.15033 ms_handle_reset on v2:172.21.15.123:6838/3252673889 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 shutdown_connections 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 1 -- 172.21.15.67:0/182917984 wait complete. 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23fb0f4b80 20 client.15033 trim_cache size 0 max 16384 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:16:50.517 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24538, 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 58.503563448999998, 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 98.880537356000005 2023-10-07T23:16:50.518 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:16:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f23d009b770 msgr2=0x7f23d009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f23d009b770 0x7f23d009dc20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000e50 tx=0x7f23dc009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 msgr2=0x7f23e8114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 0x7f23e8114fb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23e000ec30 tx=0x7f23e000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 shutdown_connections 2023-10-07T23:16:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23e80fcce0 0x7f23e810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f23d009b770 0x7f23d009dc20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23e80fe880 0x7f23e8114fb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 --2- 172.21.15.67:0/3299782437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23e80fdef0 0x7f23e810df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.511+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 >> 172.21.15.67:0/3299782437 conn(0x7f23e80f8460 msgr2=0x7f23e80fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.512+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 shutdown_connections 2023-10-07T23:16:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.512+0000 7f23e759e700 1 -- 172.21.15.67:0/3299782437 wait complete. 2023-10-07T23:16:50.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-07T23:16:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 -- 172.21.15.67:0/2277379444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fcce0 msgr2=0x7fc8fc0fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 --2- 172.21.15.67:0/2277379444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fcce0 0x7fc8fc0fd0f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0049c0 tx=0x7fc8ec009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 -- 172.21.15.67:0/2277379444 shutdown_connections 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 --2- 172.21.15.67:0/2277379444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 --2- 172.21.15.67:0/2277379444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fcce0 0x7fc8fc0fd0f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 --2- 172.21.15.67:0/2277379444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8fc0fdef0 0x7fc8fc0fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.718+0000 7fc90119d700 1 -- 172.21.15.67:0/2277379444 >> 172.21.15.67:0/2277379444 conn(0x7fc8fc0f8460 msgr2=0x7fc8fc0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.719+0000 7fc90119d700 1 -- 172.21.15.67:0/2277379444 shutdown_connections 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.719+0000 7fc90119d700 1 -- 172.21.15.67:0/2277379444 wait complete. 2023-10-07T23:16:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.719+0000 7fc90119d700 1 Processor -- start 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.719+0000 7fc90119d700 1 -- start start 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90119d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8fc0fcce0 0x7fc8fc10fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90119d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8fc0fdef0 0x7fc8fc110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90119d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90119d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1177d0 con 0x7fc8fc0fe880 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90119d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc117940 con 0x7fc8fc0fcce0 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90119d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc117b00 con 0x7fc8fc0fdef0 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc117190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44588/0 (socket says 172.21.15.67:44588) 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 -- 172.21.15.67:0/2121525745 learned_addr learned my addr 172.21.15.67:0/2121525745 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8fc0fcce0 0x7fc8fc10fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8fc0fdef0 0x7fc8fc110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 -- 172.21.15.67:0/2121525745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8fc0fdef0 msgr2=0x7fc8fc110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8fc0fdef0 0x7fc8fc110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 -- 172.21.15.67:0/2121525745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8fc0fcce0 msgr2=0x7fc8fc10fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8fc0fcce0 0x7fc8fc10fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.720+0000 7fc90099c700 1 -- 172.21.15.67:0/2121525745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec005040 con 0x7fc8fc0fe880 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.721+0000 7fc90099c700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc117190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ec30 tx=0x7fc8f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.721+0000 7fc8f8ff9700 1 -- 172.21.15.67:0/2121525745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4007b30 con 0x7fc8fc0fe880 2023-10-07T23:16:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.721+0000 7fc8f8ff9700 1 -- 172.21.15.67:0/2121525745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f400f040 con 0x7fc8fc0fe880 2023-10-07T23:16:50.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.721+0000 7fc8f8ff9700 1 -- 172.21.15.67:0/2121525745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400ca30 con 0x7fc8fc0fe880 2023-10-07T23:16:50.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.721+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc117de0 con 0x7fc8fc0fe880 2023-10-07T23:16:50.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.721+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc118270 con 0x7fc8fc0fe880 2023-10-07T23:16:50.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.724+0000 7fc8f8ff9700 1 -- 172.21.15.67:0/2121525745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fc8f401d400 con 0x7fc8fc0fe880 2023-10-07T23:16:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.725+0000 7fc910ab7b80 1 Processor -- start 2023-10-07T23:16:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.725+0000 7fc8f8ff9700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc8e409b770 0x7fc8e409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 -- start start 2023-10-07T23:16:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8f8ff9700 1 -- 172.21.15.67:0/2121525745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fc8f40c5730 con 0x7fc8fc0fe880 2023-10-07T23:16:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc8e409b770 0x7fc8e409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c626a70 con 0x562f5c6d1440 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c624270 con 0x562f5c6d1d90 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c620060 con 0x562f5c6d2cf0 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8fb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44600/0 (socket says 172.21.15.67:44600) 2023-10-07T23:16:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4083192555 learned_addr learned my addr 172.21.15.67:0/4083192555 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.726+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc8e409b770 0x7fc8e409dc20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009b80 tx=0x7fc8f000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc90099c700 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4083192555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 msgr2=0x562f5c6d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4083192555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 msgr2=0x562f5c6d26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4083192555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5c4ba270 con 0x562f5c6d1440 2023-10-07T23:16:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d1850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec013000 tx=0x7fc8ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01224612ddee8d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8e1ffb700 1 -- 172.21.15.67:0/4083192555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec00e070 con 0x562f5c6d1440 2023-10-07T23:16:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8e1ffb700 1 -- 172.21.15.67:0/4083192555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8ec000bc0 con 0x562f5c6d1440 2023-10-07T23:16:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.727+0000 7fc8e1ffb700 1 -- 172.21.15.67:0/4083192555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec013780 con 0x562f5c6d1440 2023-10-07T23:16:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4083192555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 msgr2=0x562f5c6d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d1850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec013000 tx=0x7fc8ec009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4083192555 shutdown_connections 2023-10-07T23:16:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d1850 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4083192555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4083192555 >> 172.21.15.67:0/4083192555 conn(0x562f5c6cc470 msgr2=0x562f5c6cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4083192555 shutdown_connections 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.728+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4083192555 wait complete. 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 -- start start 2023-10-07T23:16:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 Processor -- start 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1440 0x562f5c6dad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1d90 0x562f5c6db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c5cb890 con 0x562f5c6d1d90 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c5e6ea0 con 0x562f5c6d1440 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c4a2e30 con 0x562f5c6d2cf0 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.729+0000 7fc910ab7b80 10 client.0 _ll_register_callbacks cb 0x562f5c6932c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6e22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:34838/0 (socket says 172.21.15.67:34838) 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc8fb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1d90 0x562f5c6db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 -- 172.21.15.67:0/2180663951 learned_addr learned my addr 172.21.15.67:0/2180663951 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1440 0x562f5c6dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 -- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1440 msgr2=0x562f5c6dad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1440 0x562f5c6dad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 -- 172.21.15.67:0/2180663951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1d90 msgr2=0x562f5c6db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1d90 0x562f5c6db260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 -- 172.21.15.67:0/2180663951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec017040 con 0x562f5c6d2cf0 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1440 0x562f5c6dad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc90099c700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6e22c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c3a0 tx=0x7fc8f40c2d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e5eb9d9807dcf3d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc8daffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-07T23:16:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc8daffd700 1 -- 172.21.15.67:0/2180663951 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f401a2b0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f5c6c4ce0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.730+0000 7fc8daffd700 1 -- 172.21.15.67:0/2180663951 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fc8f4094fa0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc8daffd700 1 -- 172.21.15.67:0/2180663951 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f40948a0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 10 client.25099 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562f5c45cfe0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc8daffd700 1 -- 172.21.15.67:0/2180663951 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f5c6c4ce0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc8daffd700 1 -- 172.21.15.67:0/2180663951 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7fc8f40b1270 con 0x562f5c6d2cf0 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 10 client.25099 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 10 client.25099 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 20 client.25099 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c6e5e70 0x562f5c6eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 4 client.25099 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.731+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 --> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f5c6a1ab0 con 0x562f5c6e5e70 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.732+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c6e5e70 0x562f5c6eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.732+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c6e5e70 0x562f5c6eb170 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d911f895f16a0c82 server_cookie=8450f65f2d7c27ba in_seq=0 out_seq=0 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.732+0000 7fc8daffd700 10 client.25099 ms_handle_connect on v2:172.21.15.86:6838/2147538694 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc8daffd700 1 -- 172.21.15.67:0/2180663951 <== mds.0 v2:172.21.15.86:6838/2147538694 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562f5c6a1ab0 con 0x562f5c6e5e70 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc8daffd700 10 client.25099 handle_command_reply: tid=0 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 client.25099 shutdown 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 msgr2=0x562f5c6e22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6e22c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c3a0 tx=0x7fc8f40c2d90 comp rx=0 tx=0).stop 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 shutdown_connections 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c6e5e70 0x562f5c6eb170 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1440 0x562f5c6dad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6e22c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc8daffd700 0 client.25099 ms_handle_reset on v2:172.21.15.86:6838/2147538694 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/2180663951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1d90 0x562f5c6db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.733+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 >> 172.21.15.67:0/2180663951 conn(0x562f5c6cc470 msgr2=0x562f5c6c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.734+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 shutdown_connections 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.734+0000 7fc910ab7b80 1 -- 172.21.15.67:0/2180663951 wait complete. 2023-10-07T23:16:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.734+0000 7fc910ab7b80 20 client.25099 trim_cache size 0 max 16384 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 Processor -- start 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 -- start start 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c6e5600 con 0x562f5c6d1440 2023-10-07T23:16:50.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c6ccf50 con 0x562f5c6d1d90 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.751+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c6a1ab0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44618/0 (socket says 172.21.15.67:44618) 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4209861053 learned_addr learned my addr 172.21.15.67:0/4209861053 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc90099c700 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4209861053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 msgr2=0x562f5c6d8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4209861053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 msgr2=0x562f5c6d5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 -- 172.21.15.67:0/4209861053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5c5ee780 con 0x562f5c6d1440 2023-10-07T23:16:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc90099c700 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:16:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d5900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x562f5c6cd480 tx=0x7fc8ec020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae7c556e0faed66 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8e27fc700 1 -- 172.21.15.67:0/4209861053 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec00e040 con 0x562f5c6d1440 2023-10-07T23:16:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.752+0000 7fc8e27fc700 1 -- 172.21.15.67:0/4209861053 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8ec007ab0 con 0x562f5c6d1440 2023-10-07T23:16:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc8e27fc700 1 -- 172.21.15.67:0/4209861053 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec01e9a0 con 0x562f5c6d1440 2023-10-07T23:16:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4209861053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 msgr2=0x562f5c6d5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d5900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x562f5c6cd480 tx=0x7fc8ec020040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4209861053 shutdown_connections 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c6d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6d5900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/4209861053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c6d5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4209861053 >> 172.21.15.67:0/4209861053 conn(0x562f5c6cc470 msgr2=0x562f5c6cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4209861053 shutdown_connections 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.753+0000 7fc910ab7b80 1 -- 172.21.15.67:0/4209861053 wait complete. 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 -- start start 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 Processor -- start 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c735d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c738370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c738a80 con 0x562f5c6d1440 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c738bf0 con 0x562f5c6d1d90 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5c738db0 con 0x562f5c6d2cf0 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc910ab7b80 10 client.0 _ll_register_callbacks cb 0x562f5c6932c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6dfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44624/0 (socket says 172.21.15.67:44624) 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc8fbfff700 1 -- 172.21.15.67:0/175172147 learned_addr learned my addr 172.21.15.67:0/175172147 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.754+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c735d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc90099c700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c738370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8fbfff700 1 -- 172.21.15.67:0/175172147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 msgr2=0x562f5c738370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c738370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8fbfff700 1 -- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 msgr2=0x562f5c735d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:16:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c735d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8fbfff700 1 -- 172.21.15.67:0/175172147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f5c4ba270 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8fbfff700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6dfe80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009f50 tx=0x7fc8ec002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0fd35a6bbdd8da server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8daffd700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8daffd700 1 -- 172.21.15.67:0/175172147 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec00e070 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc8daffd700 1 -- 172.21.15.67:0/175172147 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fc8ec02d070 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.755+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8ec000bc0 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc8daffd700 1 -- 172.21.15.67:0/175172147 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8ec009490 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 10 client.15051 fetch_fsmap learned FSMap version 28 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562f5c5d5470 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc8daffd700 1 -- 172.21.15.67:0/175172147 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8ec000bc0 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc8daffd700 1 -- 172.21.15.67:0/175172147 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7fc8ec038070 con 0x562f5c6d1440 2023-10-07T23:16:50.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 10 client.15051 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 10 client.15051 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 20 client.15051 populate_metadata read hostname 'smithi067' 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c73d5d0 0x562f5c739f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 4 client.15051 mds_command: new command op to 24563 tid=0[{"prefix": "status"}] 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 --> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f5c5c2e00 con 0x562f5c73d5d0 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.756+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c73d5d0 0x562f5c739f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc8fb7fe700 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c73d5d0 0x562f5c739f70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9114c12b8f02f75 server_cookie=428f1eb3fa1646e7 in_seq=0 out_seq=0 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc8daffd700 10 client.15051 ms_handle_connect on v2:172.21.15.86:6838/2147538694 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc8daffd700 1 -- 172.21.15.67:0/175172147 <== mds.0 v2:172.21.15.86:6838/2147538694 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562f5c5c2e00 con 0x562f5c73d5d0 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc8daffd700 10 client.15051 handle_command_reply: tid=0 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 client.15051 shutdown 2023-10-07T23:16:50.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 msgr2=0x562f5c6dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6dfe80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009f50 tx=0x7fc8ec002040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 shutdown_connections 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f5c6d2cf0 0x562f5c738370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562f5c6d1440 0x562f5c6dfe80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x562f5c73d5d0 0x562f5c739f70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 --2- 172.21.15.67:0/175172147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f5c6d1d90 0x562f5c735d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 >> 172.21.15.67:0/175172147 conn(0x562f5c6cc470 msgr2=0x562f5c6dacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc8daffd700 0 client.15051 ms_handle_reset on v2:172.21.15.86:6838/2147538694 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 shutdown_connections 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.757+0000 7fc910ab7b80 1 -- 172.21.15.67:0/175172147 wait complete. 2023-10-07T23:16:50.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc910ab7b80 20 client.15051 trim_cache size 0 max 16384 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24563, 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 83.419853919000005, 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:16:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:16:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:16:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 90.301510946999997 2023-10-07T23:16:50.766 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc8e409b770 msgr2=0x7fc8e409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc8e409b770 0x7fc8e409dc20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009b80 tx=0x7fc8f000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 msgr2=0x7fc8fc117190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc117190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ec30 tx=0x7fc8f400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 shutdown_connections 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8fc0fcce0 0x7fc8fc10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fc8e409b770 0x7fc8e409dc20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8fc0fe880 0x7fc8fc117190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 --2- 172.21.15.67:0/2121525745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8fc0fdef0 0x7fc8fc110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:50.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 >> 172.21.15.67:0/2121525745 conn(0x7fc8fc0f8460 msgr2=0x7fc8fc0fa630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:50.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 shutdown_connections 2023-10-07T23:16:50.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:50.758+0000 7fc90119d700 1 -- 172.21.15.67:0/2121525745 wait complete. 2023-10-07T23:16:50.776 INFO:tasks.cephadm:Setup complete, yielding 2023-10-07T23:16:50.776 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-07T23:16:50.786 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T23:16:50.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-07T23:16:51.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:51 smithi067 ceph-mon[82723]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:16:51.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:51 smithi123 ceph-mon[122926]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:16:51.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:51 smithi086 ceph-mon[122131]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:16:51.558 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 -- 172.21.15.67:0/1777636682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19180704f0 msgr2=0x7f1918070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 --2- 172.21.15.67:0/1777636682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19180704f0 0x7f1918070960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f190c0049c0 tx=0x7f190c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 -- 172.21.15.67:0/1777636682 shutdown_connections 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 --2- 172.21.15.67:0/1777636682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 0x7f1918107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 --2- 172.21.15.67:0/1777636682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918071b40 0x7f191806ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 --2- 172.21.15.67:0/1777636682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19180704f0 0x7f1918070960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 -- 172.21.15.67:0/1777636682 >> 172.21.15.67:0/1777636682 conn(0x7f19180f85d0 msgr2=0x7f19180fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 -- 172.21.15.67:0/1777636682 shutdown_connections 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.008+0000 7f191d041700 1 -- 172.21.15.67:0/1777636682 wait complete. 2023-10-07T23:16:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.009+0000 7f191d041700 1 Processor -- start 2023-10-07T23:16:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.009+0000 7f191d041700 1 -- start start 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191d041700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19180704f0 0x7f191810fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191d041700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 0x7f19181102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191d041700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 0x7f1918117310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191d041700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918117950 con 0x7f1918071b40 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191d041700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918117ac0 con 0x7f1918104d70 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191d041700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1918117c80 con 0x7f19180704f0 2023-10-07T23:16:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f1916d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19180704f0 0x7f191810fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 0x7f19181102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f1916d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19180704f0 0x7f191810fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:57466/0 (socket says 172.21.15.67:57466) 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 0x7f19181102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43494/0 (socket says 172.21.15.67:43494) 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f1916d9d700 1 -- 172.21.15.67:0/2346586523 learned_addr learned my addr 172.21.15.67:0/2346586523 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191759e700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 0x7f1918117310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 -- 172.21.15.67:0/2346586523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19180704f0 msgr2=0x7f191810fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19180704f0 0x7f191810fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 -- 172.21.15.67:0/2346586523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 msgr2=0x7f1918117310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 0x7f1918117310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191659c700 1 -- 172.21.15.67:0/2346586523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c005040 con 0x7f1918071b40 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.010+0000 7f191759e700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 0x7f1918117310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:16:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.011+0000 7f191659c700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 0x7f19181102c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f190c0049c0 tx=0x7f190c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.011+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00e070 con 0x7f1918071b40 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.011+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190c004160 con 0x7f1918071b40 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.011+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1918117f00 con 0x7f1918071b40 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.011+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0123c0 con 0x7f1918071b40 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.011+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1918118380 con 0x7f1918071b40 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.012+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19181034c0 con 0x7f1918071b40 2023-10-07T23:16:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.014+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f190c01c610 con 0x7f1918071b40 2023-10-07T23:16:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.015+0000 7f18ff7fe700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f190009b7f0 0x7f190009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.015+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f190c0c63a0 con 0x7f1918071b40 2023-10-07T23:16:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.016+0000 7f1916d9d700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f190009b7f0 0x7f190009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:53.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.016+0000 7f1916d9d700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f190009b7f0 0x7f190009dca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1918111270 tx=0x7f190800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:53.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.019+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f190c0918b0 con 0x7f1918071b40 2023-10-07T23:16:53.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.241+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f191802b3c0 con 0x7f190009b7f0 2023-10-07T23:16:53.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.242+0000 7f18ff7fe700 1 -- 172.21.15.67:0/2346586523 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f191802b3c0 con 0x7f190009b7f0 2023-10-07T23:16:53.242 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2023-10-07T23:16:53.243 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2023-10-07T23:16:53.243 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f190009b7f0 msgr2=0x7f190009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f190009b7f0 0x7f190009dca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1918111270 tx=0x7f190800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 msgr2=0x7f19181102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 0x7f19181102c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f190c0049c0 tx=0x7f190c009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 shutdown_connections 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1918104d70 0x7f1918117310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f190009b7f0 0x7f190009dca0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19180704f0 0x7f191810fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 --2- 172.21.15.67:0/2346586523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1918071b40 0x7f19181102c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:53.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 >> 172.21.15.67:0/2346586523 conn(0x7f19180f85d0 msgr2=0x7f19180fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:53.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 shutdown_connections 2023-10-07T23:16:53.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:53.244+0000 7f191d041700 1 -- 172.21.15.67:0/2346586523 wait complete. 2023-10-07T23:16:53.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:53 smithi123 ceph-mon[122926]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:53.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:53 smithi086 ceph-mon[122131]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:53.454 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:53 smithi067 ceph-mon[82723]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:53.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-07T23:16:54.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:54 smithi123 ceph-mon[122926]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:16:54.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:54 smithi086 ceph-mon[122131]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:16:54.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:54 smithi067 ceph-mon[82723]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:16:54.512 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:55.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:55 smithi123 ceph-mon[122926]: pgmap v334: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:16:55.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:55 smithi086 ceph-mon[122131]: pgmap v334: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:16:55.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:55 smithi067 ceph-mon[82723]: pgmap v334: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:16:55.917 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 -- 172.21.15.67:0/2230107319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0071b20 msgr2=0x7f86e0070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/2230107319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0071b20 0x7f86e0070030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f86d00049c0 tx=0x7f86d0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 -- 172.21.15.67:0/2230107319 shutdown_connections 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/2230107319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e0104d80 0x7f86e01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/2230107319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0071b20 0x7f86e0070030 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/2230107319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e0070570 0x7f86e00709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.916+0000 7f86e4f1c700 1 -- 172.21.15.67:0/2230107319 >> 172.21.15.67:0/2230107319 conn(0x7f86e00f85c0 msgr2=0x7f86e00fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.917+0000 7f86e4f1c700 1 -- 172.21.15.67:0/2230107319 shutdown_connections 2023-10-07T23:16:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.917+0000 7f86e4f1c700 1 -- 172.21.15.67:0/2230107319 wait complete. 2023-10-07T23:16:55.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.917+0000 7f86e4f1c700 1 Processor -- start 2023-10-07T23:16:55.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 -- start start 2023-10-07T23:16:55.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 0x7f86e010fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:55.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e0071b20 0x7f86e0110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e0104d80 0x7f86e01172b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01178f0 con 0x7f86e0070570 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0117a60 con 0x7f86e0104d80 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86e4f1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0117c20 con 0x7f86e0071b20 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86de59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 0x7f86e010fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86de59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 0x7f86e010fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43518/0 (socket says 172.21.15.67:43518) 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86de59c700 1 -- 172.21.15.67:0/28474552 learned_addr learned my addr 172.21.15.67:0/28474552 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86ded9d700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e0104d80 0x7f86e01172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.918+0000 7f86ddd9b700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e0071b20 0x7f86e0110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:55.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86de59c700 1 -- 172.21.15.67:0/28474552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e0071b20 msgr2=0x7f86e0110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86de59c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e0071b20 0x7f86e0110260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86de59c700 1 -- 172.21.15.67:0/28474552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e0104d80 msgr2=0x7f86e01172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86de59c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e0104d80 0x7f86e01172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86de59c700 1 -- 172.21.15.67:0/28474552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0005040 con 0x7f86e0070570 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86de59c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 0x7f86e010fd20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86d00137e0 tx=0x7f86d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000e050 con 0x7f86e0070570 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d00097b0 con 0x7f86e0070570 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e0117ea0 con 0x7f86e0070570 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d001be40 con 0x7f86e0070570 2023-10-07T23:16:55.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.919+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e01182c0 con 0x7f86e0070570 2023-10-07T23:16:55.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.921+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e01034d0 con 0x7f86e0070570 2023-10-07T23:16:55.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.922+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f86d001b4f0 con 0x7f86e0070570 2023-10-07T23:16:55.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.924+0000 7f86ceffd700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f86c809b770 0x7f86c809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:55.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.924+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f86d00c6910 con 0x7f86e0070570 2023-10-07T23:16:55.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.924+0000 7f86ddd9b700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f86c809b770 0x7f86c809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:55.925 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.924+0000 7f86ddd9b700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f86c809b770 0x7f86c809dc20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009b80 tx=0x7f86d400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:55.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:55.928+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f86d0091dc0 con 0x7f86e0070570 2023-10-07T23:16:56.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.141+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f86e002b370 con 0x7f86c809b770 2023-10-07T23:16:56.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.154+0000 7f86ceffd700 1 -- 172.21.15.67:0/28474552 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4030 (secure 0 0 0) 0x7f86e002b370 con 0x7f86c809b770 2023-10-07T23:16:56.155 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-07T23:16:56.155 INFO:teuthology.orchestra.run.smithi067.stdout:mds.a smithi067 running (98s) 19s ago 98s 15.1M - 17.2.6-1320-g08cac018 5b7bdd600973 0aa2dc462957 2023-10-07T23:16:56.155 INFO:teuthology.orchestra.run.smithi067.stdout:mds.b smithi086 running (96s) 25s ago 95s 25.6M - 17.2.6-1320-g08cac018 5b7bdd600973 807db558f59c 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.c smithi123 running (110s) 25s ago 109s 16.7M - 17.2.6-1320-g08cac018 5b7bdd600973 27399760f4fb 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.d smithi067 running (117s) 19s ago 117s 16.4M - 17.2.6-1320-g08cac018 5b7bdd600973 94fc296308b7 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.e smithi086 running (115s) 25s ago 114s 16.5M - 17.2.6-1320-g08cac018 5b7bdd600973 3af8eaca59f3 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.f smithi123 running (112s) 25s ago 112s 17.8M - 17.2.6-1320-g08cac018 5b7bdd600973 26a37b7ad3c8 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.g smithi067 running (101s) 19s ago 101s 12.7M - 17.2.6-1320-g08cac018 5b7bdd600973 ad638778567c 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.h smithi086 running (107s) 25s ago 106s 18.9M - 17.2.6-1320-g08cac018 5b7bdd600973 a57542ed04d8 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.i smithi123 running (104s) 25s ago 104s 16.8M - 17.2.6-1320-g08cac018 5b7bdd600973 f6a39fbd68b0 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.j smithi067 running (2m) 19s ago 2m 12.9M - 17.2.6-1320-g08cac018 5b7bdd600973 c7b6b3ba9641 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.k smithi086 running (2m) 25s ago 2m 16.2M - 17.2.6-1320-g08cac018 5b7bdd600973 e9decf3230a8 2023-10-07T23:16:56.156 INFO:teuthology.orchestra.run.smithi067.stdout:mds.l smithi123 running (2m) 25s ago 2m 17.6M - 17.2.6-1320-g08cac018 5b7bdd600973 baede428ce26 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi067 *:9283 running (9m) 19s ago 9m 512M - 17.2.6-1320-g08cac018 5b7bdd600973 ce6eb291ec33 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi086 *:8443 running (7m) 25s ago 7m 441M - 17.2.6-1320-g08cac018 5b7bdd600973 670d3a75a356 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi123 *:8443 running (7m) 25s ago 7m 441M - 17.2.6-1320-g08cac018 5b7bdd600973 8224dbad7637 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (9m) 19s ago 9m 91.8M 2048M 17.2.6-1320-g08cac018 5b7bdd600973 e8557d0be314 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi086 running (7m) 25s ago 7m 68.8M 2048M 17.2.6-1320-g08cac018 5b7bdd600973 5f35a1f401f9 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi123 running (7m) 25s ago 7m 62.3M 2048M 17.2.6-1320-g08cac018 5b7bdd600973 43a316b4d952 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (7m) 19s ago 7m 72.1M 4316M 17.2.6-1320-g08cac018 5b7bdd600973 a9e33f6e9e4e 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi086 running (6m) 25s ago 6m 76.5M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 85f1e3f1b300 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi123 running (6m) 25s ago 6m 73.9M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 1b52981d523f 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 smithi067 running (5m) 19s ago 5m 66.8M 4316M 17.2.6-1320-g08cac018 5b7bdd600973 ad1f0690f7dc 2023-10-07T23:16:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 smithi086 running (5m) 25s ago 5m 66.3M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 1954dbbb228c 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 smithi123 running (4m) 25s ago 4m 70.8M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 86946c4ca7df 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 smithi067 running (4m) 19s ago 4m 61.7M 4316M 17.2.6-1320-g08cac018 5b7bdd600973 e1dc9402c576 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 smithi086 running (4m) 25s ago 4m 67.7M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 1de9d179cf2b 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 smithi123 running (3m) 25s ago 3m 61.9M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 59204a5354db 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 smithi067 running (3m) 19s ago 3m 59.7M 4316M 17.2.6-1320-g08cac018 5b7bdd600973 1453a8cc8324 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 smithi086 running (2m) 25s ago 2m 60.4M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 ad12e10e42f8 2023-10-07T23:16:56.158 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 smithi123 running (2m) 25s ago 2m 54.2M 4293M 17.2.6-1320-g08cac018 5b7bdd600973 9ada03f59586 2023-10-07T23:16:56.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f86c809b770 msgr2=0x7f86c809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:56.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f86c809b770 0x7f86c809dc20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009b80 tx=0x7f86d400b040 comp rx=0 tx=0).stop 2023-10-07T23:16:56.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 msgr2=0x7f86e010fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:56.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 0x7f86e010fd20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86d00137e0 tx=0x7f86d0009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:56.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 shutdown_connections 2023-10-07T23:16:56.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e0104d80 0x7f86e01172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:56.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f86c809b770 0x7f86c809dc20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:56.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86e0070570 0x7f86e010fd20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:56.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 --2- 172.21.15.67:0/28474552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e0071b20 0x7f86e0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:56.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 >> 172.21.15.67:0/28474552 conn(0x7f86e00f85c0 msgr2=0x7f86e0106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:56.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 shutdown_connections 2023-10-07T23:16:56.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:56.157+0000 7f86e4f1c700 1 -- 172.21.15.67:0/28474552 wait complete. 2023-10-07T23:16:56.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-07T23:16:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:16:57.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:57 smithi123 ceph-mon[122926]: pgmap v335: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:57.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:57 smithi123 ceph-mon[122926]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:16:57.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:57 smithi086 ceph-mon[122131]: pgmap v335: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:57.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:57 smithi086 ceph-mon[122131]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:16:57.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:57 smithi067 ceph-mon[82723]: pgmap v335: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:16:57.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:57 smithi067 ceph-mon[82723]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:16:58.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 -- 172.21.15.67:0/809849803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fdfd0 msgr2=0x7f05c00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:58.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 --2- 172.21.15.67:0/809849803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fdfd0 0x7f05c00fe440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05b40049c0 tx=0x7f05b4009e30 comp rx=0 tx=0).stop 2023-10-07T23:16:58.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 -- 172.21.15.67:0/809849803 shutdown_connections 2023-10-07T23:16:58.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 --2- 172.21.15.67:0/809849803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 0x7f05c00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:58.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 --2- 172.21.15.67:0/809849803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05c00fe980 0x7f05c01052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:58.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 --2- 172.21.15.67:0/809849803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fdfd0 0x7f05c00fe440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:58.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 -- 172.21.15.67:0/809849803 >> 172.21.15.67:0/809849803 conn(0x7f05c00f85c0 msgr2=0x7f05c00fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:58.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 -- 172.21.15.67:0/809849803 shutdown_connections 2023-10-07T23:16:58.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.861+0000 7f05c63df700 1 -- 172.21.15.67:0/809849803 wait complete. 2023-10-07T23:16:58.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.862+0000 7f05c63df700 1 Processor -- start 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.862+0000 7f05c63df700 1 -- start start 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c63df700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 0x7f05c010fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c63df700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05c00fdfd0 0x7f05c01101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c63df700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fe980 0x7f05c0117240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c63df700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0117890 con 0x7f05c00fe980 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c63df700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0117a00 con 0x7f05c00fce60 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c63df700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0117bc0 con 0x7f05c00fdfd0 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05bffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 0x7f05c010fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05bffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 0x7f05c010fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:46646/0 (socket says 172.21.15.67:46646) 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05bffff700 1 -- 172.21.15.67:0/3293657553 learned_addr learned my addr 172.21.15.67:0/3293657553 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:16:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c497c700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fe980 0x7f05c0117240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05bf7fe700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05c00fdfd0 0x7f05c01101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c497c700 1 -- 172.21.15.67:0/3293657553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05c00fdfd0 msgr2=0x7f05c01101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c497c700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05c00fdfd0 0x7f05c01101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c497c700 1 -- 172.21.15.67:0/3293657553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 msgr2=0x7f05c010fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c497c700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 0x7f05c010fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.863+0000 7f05c497c700 1 -- 172.21.15.67:0/3293657553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4005040 con 0x7f05c00fe980 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.864+0000 7f05c497c700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fe980 0x7f05c0117240 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05b800ec20 tx=0x7f05b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.864+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8007b30 con 0x7f05c00fe980 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.864+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b800f040 con 0x7f05c00fe980 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.864+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c0117ea0 con 0x7f05c00fe980 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.864+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800c9f0 con 0x7f05c00fe980 2023-10-07T23:16:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.864+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c0118380 con 0x7f05c00fe980 2023-10-07T23:16:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.865+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c0108670 con 0x7f05c00fe980 2023-10-07T23:16:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.867+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f05b801d400 con 0x7f05c00fe980 2023-10-07T23:16:58.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.868+0000 7f05bcff9700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f05a809b7f0 0x7f05a809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:16:58.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.869+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f05b80c5580 con 0x7f05c00fe980 2023-10-07T23:16:58.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.869+0000 7f05bffff700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f05a809b7f0 0x7f05a809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:16:58.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.869+0000 7f05bffff700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f05a809b7f0 0x7f05a809dca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05c0110b70 tx=0x7f05b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:16:58.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:58.872+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f05b8090a30 con 0x7f05c00fe980 2023-10-07T23:16:59.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.087+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f05c0104f70 con 0x7f05a809b7f0 2023-10-07T23:16:59.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.093+0000 7f05bcff9700 1 -- 172.21.15.67:0/3293657553 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f05c0104f70 con 0x7f05a809b7f0 2023-10-07T23:16:59.092 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-07T23:16:59.092 INFO:teuthology.orchestra.run.smithi067.stdout:mds.all 12/12 28s ago 2m smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-10-07T23:16:59.092 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 28s ago 7m smithi067=x;smithi086=y;smithi123=z;count:3 2023-10-07T23:16:59.093 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 28s ago 7m smithi067:172.21.15.67=a;smithi086:172.21.15.86=b;smithi123:172.21.15.123=c;count:3 2023-10-07T23:16:59.093 INFO:teuthology.orchestra.run.smithi067.stdout:osd 12 28s ago - 2023-10-07T23:16:59.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f05a809b7f0 msgr2=0x7f05a809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:59.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f05a809b7f0 0x7f05a809dca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05c0110b70 tx=0x7f05b000b040 comp rx=0 tx=0).stop 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fe980 msgr2=0x7f05c0117240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fe980 0x7f05c0117240 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05b800ec20 tx=0x7f05b800b040 comp rx=0 tx=0).stop 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 shutdown_connections 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05c00fce60 0x7f05c010fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f05a809b7f0 0x7f05a809dca0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05c00fe980 0x7f05c0117240 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 --2- 172.21.15.67:0/3293657553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05c00fdfd0 0x7f05c01101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 >> 172.21.15.67:0/3293657553 conn(0x7f05c00f85c0 msgr2=0x7f05c0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 shutdown_connections 2023-10-07T23:16:59.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:16:59.095+0000 7f05c63df700 1 -- 172.21.15.67:0/3293657553 wait complete. 2023-10-07T23:16:59.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:16:59 smithi123 ceph-mon[122926]: pgmap v336: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:16:59.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:16:59 smithi086 ceph-mon[122131]: pgmap v336: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:16:59.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:16:59 smithi067 ceph-mon[82723]: pgmap v336: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:16:59.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-07T23:17:00.319 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:17:00.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:00 smithi123 ceph-mon[122926]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:00.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:00 smithi086 ceph-mon[122131]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:00.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:00 smithi067 ceph-mon[82723]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:01.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:01 smithi067 ceph-mon[82723]: pgmap v337: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:01.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:01 smithi123 ceph-mon[122926]: pgmap v337: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:01.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:01 smithi086 ceph-mon[122131]: pgmap v337: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.724+0000 7fde20b46700 1 -- 172.21.15.67:0/2460079629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 msgr2=0x7fde1c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.724+0000 7fde20b46700 1 --2- 172.21.15.67:0/2460079629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c0fd270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0049c0 tx=0x7fde0c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 -- 172.21.15.67:0/2460079629 shutdown_connections 2023-10-07T23:17:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 --2- 172.21.15.67:0/2460079629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1c0fe980 0x7fde1c1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 --2- 172.21.15.67:0/2460079629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c0fd270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 --2- 172.21.15.67:0/2460079629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde1c0fdfd0 0x7fde1c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 -- 172.21.15.67:0/2460079629 >> 172.21.15.67:0/2460079629 conn(0x7fde1c0f85a0 msgr2=0x7fde1c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 -- 172.21.15.67:0/2460079629 shutdown_connections 2023-10-07T23:17:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 -- 172.21.15.67:0/2460079629 wait complete. 2023-10-07T23:17:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.725+0000 7fde20b46700 1 Processor -- start 2023-10-07T23:17:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 -- start start 2023-10-07T23:17:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1c0fdfd0 0x7fde1c06ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde1c0fe980 0x7fde1c0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c070c40 con 0x7fde1c0fce60 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c070db0 con 0x7fde1c0fe980 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde20b46700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c117bd0 con 0x7fde1c0fdfd0 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde1a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde1a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43554/0 (socket says 172.21.15.67:43554) 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde1a59c700 1 -- 172.21.15.67:0/777180730 learned_addr learned my addr 172.21.15.67:0/777180730 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde1ad9d700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde1c0fe980 0x7fde1c0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.726+0000 7fde19d9b700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1c0fdfd0 0x7fde1c06ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:01.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde1a59c700 1 -- 172.21.15.67:0/777180730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1c0fdfd0 msgr2=0x7fde1c06ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde1a59c700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1c0fdfd0 0x7fde1c06ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde1a59c700 1 -- 172.21.15.67:0/777180730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde1c0fe980 msgr2=0x7fde1c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde1a59c700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde1c0fe980 0x7fde1c0704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde1a59c700 1 -- 172.21.15.67:0/777180730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c005040 con 0x7fde1c0fce60 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde1a59c700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c071930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009f50 tx=0x7fde0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00e040 con 0x7fde1c0fce60 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0c0096f0 con 0x7fde1c0fce60 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c117e20 con 0x7fde1c0fce60 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c013e80 con 0x7fde1c0fce60 2023-10-07T23:17:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.727+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c1181d0 con 0x7fde1c0fce60 2023-10-07T23:17:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.728+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c1042d0 con 0x7fde1c0fce60 2023-10-07T23:17:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.730+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fde0c013520 con 0x7fde1c0fce60 2023-10-07T23:17:01.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.732+0000 7fde0affd700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fde0409b7f0 0x7fde0409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:01.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.732+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fde0c0c6f20 con 0x7fde1c0fce60 2023-10-07T23:17:01.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.732+0000 7fde19d9b700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fde0409b7f0 0x7fde0409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:01.733 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.732+0000 7fde19d9b700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fde0409b7f0 0x7fde0409dca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde10009fd0 tx=0x7fde1000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:01.734 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.734+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fde0c092450 con 0x7fde1c0fce60 2023-10-07T23:17:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.942+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fde1c02b460 con 0x7fde0409b7f0 2023-10-07T23:17:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.944+0000 7fde0affd700 1 -- 172.21.15.67:0/777180730 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fde1c02b460 con 0x7fde0409b7f0 2023-10-07T23:17:01.945 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2023-10-07T23:17:01.945 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2023-10-07T23:17:01.945 INFO:teuthology.orchestra.run.smithi067.stdout:smithi086 172.21.15.86 2023-10-07T23:17:01.945 INFO:teuthology.orchestra.run.smithi067.stdout:smithi123 172.21.15.123 2023-10-07T23:17:01.945 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2023-10-07T23:17:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.946+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fde0409b7f0 msgr2=0x7fde0409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.946+0000 7fde20b46700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fde0409b7f0 0x7fde0409dca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde10009fd0 tx=0x7fde1000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.946+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 msgr2=0x7fde1c071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.946+0000 7fde20b46700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c071930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009f50 tx=0x7fde0c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 shutdown_connections 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde1c0fe980 0x7fde1c0704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fde0409b7f0 0x7fde0409dca0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c0fce60 0x7fde1c071930 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 --2- 172.21.15.67:0/777180730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1c0fdfd0 0x7fde1c06ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 >> 172.21.15.67:0/777180730 conn(0x7fde1c0f85a0 msgr2=0x7fde1c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 shutdown_connections 2023-10-07T23:17:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:01.947+0000 7fde20b46700 1 -- 172.21.15.67:0/777180730 wait complete. 2023-10-07T23:17:02.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-07T23:17:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:17:03.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:03 smithi123 ceph-mon[122926]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:03.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:03 smithi123 ceph-mon[122926]: pgmap v338: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:03.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:03 smithi067 ceph-mon[82723]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:03.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:03 smithi067 ceph-mon[82723]: pgmap v338: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:03.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:03 smithi086 ceph-mon[122131]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:03.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:03 smithi086 ceph-mon[122131]: pgmap v338: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 -- 172.21.15.67:0/2680995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fdfd0 msgr2=0x7f4f240fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 --2- 172.21.15.67:0/2680995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fdfd0 0x7f4f240fe440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f18004900 tx=0x7f4f18009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 -- 172.21.15.67:0/2680995499 shutdown_connections 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 --2- 172.21.15.67:0/2680995499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f240fe980 0x7f4f241052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 --2- 172.21.15.67:0/2680995499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fce60 0x7f4f240fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 --2- 172.21.15.67:0/2680995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fdfd0 0x7f4f240fe440 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 -- 172.21.15.67:0/2680995499 >> 172.21.15.67:0/2680995499 conn(0x7f4f240f85c0 msgr2=0x7f4f240fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 -- 172.21.15.67:0/2680995499 shutdown_connections 2023-10-07T23:17:04.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.558+0000 7f4f2905c700 1 -- 172.21.15.67:0/2680995499 wait complete. 2023-10-07T23:17:04.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 Processor -- start 2023-10-07T23:17:04.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 -- start start 2023-10-07T23:17:04.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fce60 0x7f4f24071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:04.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 0x7f4f2406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:04.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f240fe980 0x7f4f240704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24070c40 con 0x7f4f240fe980 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24070db0 con 0x7f4f240fce60 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.559+0000 7f4f2905c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24117b70 con 0x7f4f240fdfd0 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 0x7f4f2406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 0x7f4f2406ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:44934/0 (socket says 172.21.15.67:44934) 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 -- 172.21.15.67:0/578265080 learned_addr learned my addr 172.21.15.67:0/578265080 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f22d9d700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fce60 0x7f4f24071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2359e700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f240fe980 0x7f4f240704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 -- 172.21.15.67:0/578265080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fce60 msgr2=0x7f4f24071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fce60 0x7f4f24071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 -- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f240fe980 msgr2=0x7f4f240704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f240fe980 0x7f4f240704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 -- 172.21.15.67:0/578265080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18005040 con 0x7f4f240fdfd0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.560+0000 7f4f2259c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 0x7f4f2406ff90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009c90 tx=0x7f4f18009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.561+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1800e040 con 0x7f4f240fdfd0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.561+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1801c430 con 0x7f4f240fdfd0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.561+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f24117dc0 con 0x7f4f240fdfd0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.561+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18012630 con 0x7f4f240fdfd0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.561+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f241181f0 con 0x7f4f240fdfd0 2023-10-07T23:17:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.562+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f24108670 con 0x7f4f240fdfd0 2023-10-07T23:17:04.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.564+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f4f1801c5a0 con 0x7f4f240fdfd0 2023-10-07T23:17:04.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.565+0000 7f4f0b7fe700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4f0c09b770 0x7f4f0c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:04.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.565+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f4f180c7180 con 0x7f4f240fdfd0 2023-10-07T23:17:04.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.565+0000 7f4f22d9d700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4f0c09b770 0x7f4f0c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:04.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.566+0000 7f4f22d9d700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4f0c09b770 0x7f4f0c09dc20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009b80 tx=0x7f4f1400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:04.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.567+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4f180926e0 con 0x7f4f240fdfd0 2023-10-07T23:17:04.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.776+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f2405e080 con 0x7f4f0c09b770 2023-10-07T23:17:04.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.778+0000 7f4f0b7fe700 1 -- 172.21.15.67:0/578265080 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f4f2405e080 con 0x7f4f0c09b770 2023-10-07T23:17:04.778 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-07T23:17:04.778 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-10-07T23:17:04.778 INFO:teuthology.orchestra.run.smithi067.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-07T23:17:04.779 INFO:teuthology.orchestra.run.smithi067.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4f0c09b770 msgr2=0x7f4f0c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4f0c09b770 0x7f4f0c09dc20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009b80 tx=0x7f4f1400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 msgr2=0x7f4f2406ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 0x7f4f2406ff90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009c90 tx=0x7f4f18009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 shutdown_connections 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f240fce60 0x7f4f24071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4f0c09b770 0x7f4f0c09dc20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f240fe980 0x7f4f240704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 --2- 172.21.15.67:0/578265080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f240fdfd0 0x7f4f2406ff90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:04.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 >> 172.21.15.67:0/578265080 conn(0x7f4f240f85c0 msgr2=0x7f4f240fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:04.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 shutdown_connections 2023-10-07T23:17:04.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:04.780+0000 7f4f2905c700 1 -- 172.21.15.67:0/578265080 wait complete. 2023-10-07T23:17:05.104 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:04 smithi067 ceph-mon[82723]: pgmap v339: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:17:05.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:04 smithi123 ceph-mon[122926]: pgmap v339: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:17:05.164 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-07T23:17:05.173 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T23:17:05.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-07T23:17:05.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:04 smithi086 ceph-mon[122131]: pgmap v339: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:17:05.964 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:17:06.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:05 smithi123 ceph-mon[122926]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:06.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:05 smithi086 ceph-mon[122131]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:06.233 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:05 smithi067 ceph-mon[82723]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-07T23:17:07.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:06 smithi123 ceph-mon[122926]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:07.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:06 smithi067 ceph-mon[82723]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:07.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:06 smithi086 ceph-mon[122131]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:07.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 -- 172.21.15.67:0/1526402215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 msgr2=0x7f42140ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 --2- 172.21.15.67:0/1526402215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f42140ffd70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f42040049c0 tx=0x7f4204009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 -- 172.21.15.67:0/1526402215 shutdown_connections 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 --2- 172.21.15.67:0/1526402215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4214102bd0 0x7f4214104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 --2- 172.21.15.67:0/1526402215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f42140ffd70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 --2- 172.21.15.67:0/1526402215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42141002b0 0x7f4214102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 -- 172.21.15.67:0/1526402215 >> 172.21.15.67:0/1526402215 conn(0x7f42140f74b0 msgr2=0x7f42140f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 -- 172.21.15.67:0/1526402215 shutdown_connections 2023-10-07T23:17:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.407+0000 7f42189a6700 1 -- 172.21.15.67:0/1526402215 wait complete. 2023-10-07T23:17:07.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.408+0000 7f42189a6700 1 Processor -- start 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.408+0000 7f42189a6700 1 -- start start 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f42189a6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f421410fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f42189a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42141002b0 0x7f42141101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f42189a6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4214102bd0 0x7f42141171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f42189a6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214117830 con 0x7f42140fd990 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f42189a6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42141179a0 con 0x7f4214102bd0 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f42189a6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214117b60 con 0x7f42141002b0 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f421410fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f421410fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60522/0 (socket says 172.21.15.67:60522) 2023-10-07T23:17:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 -- 172.21.15.67:0/2427102232 learned_addr learned my addr 172.21.15.67:0/2427102232 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f4211d9b700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42141002b0 0x7f42141101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f4212d9d700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4214102bd0 0x7f42141171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 -- 172.21.15.67:0/2427102232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42141002b0 msgr2=0x7f42141101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42141002b0 0x7f42141101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 -- 172.21.15.67:0/2427102232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4214102bd0 msgr2=0x7f42141171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4214102bd0 0x7f42141171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.409+0000 7f421259c700 1 -- 172.21.15.67:0/2427102232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204005040 con 0x7f42140fd990 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.410+0000 7f421259c700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f421410fc60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f420400b3b0 tx=0x7f4204009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.410+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400e040 con 0x7f42140fd990 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.410+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4204009670 con 0x7f42140fd990 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.410+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4214117de0 con 0x7f42140fd990 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.410+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204013d50 con 0x7f42140fd990 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.410+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4214118250 con 0x7f42140fd990 2023-10-07T23:17:07.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.411+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4214104700 con 0x7f42140fd990 2023-10-07T23:17:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.413+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f42040091a0 con 0x7f42140fd990 2023-10-07T23:17:07.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.414+0000 7f4202ffd700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41fc09b7f0 0x7f41fc09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:07.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.414+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f42040c6420 con 0x7f42140fd990 2023-10-07T23:17:07.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.415+0000 7f4211d9b700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41fc09b7f0 0x7f41fc09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:07.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.415+0000 7f4211d9b700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41fc09b7f0 0x7f41fc09dca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4208004f40 tx=0x7f420800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:07.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.418+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f42040918d0 con 0x7f42140fd990 2023-10-07T23:17:07.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.668+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f421402b350 con 0x7f42140fd990 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.668+0000 7f4202ffd700 1 -- 172.21.15.67:0/2427102232 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2889 (secure 0 0 0) 0x7f4204094ef0 con 0x7f42140fd990 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout:e28 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout:legacy client fscid: 1 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout:Filesystem 'cephfs' (1) 2023-10-07T23:17:07.669 INFO:teuthology.orchestra.run.smithi067.stdout:fs_name cephfs 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 28 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-10-07T23:15:26.869397+0000 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2023-10-07T23:15:55.854427+0000 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:tableserver 0 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:root 0 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:session_timeout 300 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:session_autoclose 300 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:max_file_size 1099511627776 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:required_client_features {} 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure 0 2023-10-07T23:17:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure_osd_epoch 0 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:max_mds 5 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:in 0,1,2,3,4 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:up {0=24563,1=24538,2=24521,3=24467,4=24448} 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:failed 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:damaged 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:stopped 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:data_pools [3,4] 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:metadata_pool 2 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:inline_data disabled 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:balancer 2023-10-07T23:17:07.671 INFO:teuthology.orchestra.run.smithi067.stdout:standby_count_wanted 0 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b{0:24563} state up:active seq 3 addr [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c{0:24520} state up:standby-replay seq 1 addr [v2:172.21.15.123:6836/1647886755,v1:172.21.15.123:6837/1647886755] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.i{1:24538} state up:active seq 12 addr [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.f{1:24508} state up:standby-replay seq 1 addr [v2:172.21.15.123:6834/4209275840,v1:172.21.15.123:6835/4209275840] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.h{2:24521} state up:active seq 12 addr [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.e{2:24482} state up:standby-replay seq 1 addr [v2:172.21.15.86:6834/3322109112,v1:172.21.15.86:6835/3322109112] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d{3:24467} state up:active seq 15 addr [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.k{3:24443} state up:standby-replay seq 1 addr [v2:172.21.15.86:6832/1133310774,v1:172.21.15.86:6833/1133310774] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.l{4:24448} state up:active seq 18 addr [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a{4:14502} state up:standby-replay seq 1 addr [v2:172.21.15.67:6840/3010063667,v1:172.21.15.67:6841/3010063667] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:07.672 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:07.673 INFO:teuthology.orchestra.run.smithi067.stdout:Standby daemons: 2023-10-07T23:17:07.673 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:07.673 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.j{-1:14421} state up:standby seq 1 addr [v2:172.21.15.67:6834/959405817,v1:172.21.15.67:6835/959405817] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.673 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.g{-1:14484} state up:standby seq 1 addr [v2:172.21.15.67:6838/3904846086,v1:172.21.15.67:6839/3904846086] compat {c=[1],r=[1],i=[7ff]}] 2023-10-07T23:17:07.673 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41fc09b7f0 msgr2=0x7f41fc09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41fc09b7f0 0x7f41fc09dca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4208004f40 tx=0x7f420800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 msgr2=0x7f421410fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f421410fc60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f420400b3b0 tx=0x7f4204009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 shutdown_connections 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4214102bd0 0x7f42141171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f41fc09b7f0 0x7f41fc09dca0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140fd990 0x7f421410fc60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 --2- 172.21.15.67:0/2427102232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42141002b0 0x7f42141101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.670+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 >> 172.21.15.67:0/2427102232 conn(0x7f42140f74b0 msgr2=0x7f42140fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.671+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 shutdown_connections 2023-10-07T23:17:07.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:07.671+0000 7f42189a6700 1 -- 172.21.15.67:0/2427102232 wait complete. 2023-10-07T23:17:07.675 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:17:08.004 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:07 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2427102232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-07T23:17:08.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-07T23:17:08.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:07 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2427102232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-07T23:17:08.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:07 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2427102232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-07T23:17:08.822 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:17:09.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:08 smithi067 ceph-mon[82723]: pgmap v341: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:17:09.140 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:08 smithi123 ceph-mon[122926]: pgmap v341: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:17:09.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:08 smithi086 ceph-mon[122131]: pgmap v341: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 -- 172.21.15.67:0/1318899889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2680ff060 msgr2=0x7fd2680ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 --2- 172.21.15.67:0/1318899889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2680ff060 0x7fd2680ff470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd2580049c0 tx=0x7fd258009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 -- 172.21.15.67:0/1318899889 shutdown_connections 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 --2- 172.21.15.67:0/1318899889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100c00 0x7fd2681074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 --2- 172.21.15.67:0/1318899889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2680ff060 0x7fd2680ff470 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 --2- 172.21.15.67:0/1318899889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd268100270 0x7fd2681006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 -- 172.21.15.67:0/1318899889 >> 172.21.15.67:0/1318899889 conn(0x7fd2680fa7c0 msgr2=0x7fd2680fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 -- 172.21.15.67:0/1318899889 shutdown_connections 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.374+0000 7fd26d335700 1 -- 172.21.15.67:0/1318899889 wait complete. 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 Processor -- start 2023-10-07T23:17:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 -- start start 2023-10-07T23:17:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2680ff060 0x7fd26810fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 0x7fd2681102e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd268100c00 0x7fd268117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268117970 con 0x7fd268100c00 2023-10-07T23:17:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268117ae0 con 0x7fd268100270 2023-10-07T23:17:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.375+0000 7fd26d335700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd268117ca0 con 0x7fd2680ff060 2023-10-07T23:17:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2677fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd268100c00 0x7fd268117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 0x7fd2681102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 0x7fd2681102e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:47182/0 (socket says 172.21.15.67:47182) 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 -- 172.21.15.67:0/604734210 learned_addr learned my addr 172.21.15.67:0/604734210 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd266ffd700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2680ff060 0x7fd26810fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 -- 172.21.15.67:0/604734210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2680ff060 msgr2=0x7fd26810fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2680ff060 0x7fd26810fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 -- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd268100c00 msgr2=0x7fd268117330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd268100c00 0x7fd268117330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 -- 172.21.15.67:0/604734210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600048f0 con 0x7fd268100270 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2677fe700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd268100c00 0x7fd268117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.376+0000 7fd2667fc700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 0x7fd2681102e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00ebc0 tx=0x7fd25c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.377+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c007cc0 con 0x7fd268100270 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.377+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd25c00f040 con 0x7fd268100270 2023-10-07T23:17:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.377+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd258005040 con 0x7fd268100270 2023-10-07T23:17:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.377+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00c880 con 0x7fd268100270 2023-10-07T23:17:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.377+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd268118240 con 0x7fd268100270 2023-10-07T23:17:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.378+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2681065d0 con 0x7fd268100270 2023-10-07T23:17:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.380+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd25c00ccb0 con 0x7fd268100270 2023-10-07T23:17:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.381+0000 7fd24f7fe700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd25009b770 0x7fd25009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.381+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fd25c0c4dc0 con 0x7fd268100270 2023-10-07T23:17:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.381+0000 7fd266ffd700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd25009b770 0x7fd25009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.382+0000 7fd266ffd700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd25009b770 0x7fd25009dc20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd258004890 tx=0x7fd258009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.383+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd25c090270 con 0x7fd268100270 2023-10-07T23:17:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.588+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fd26802b490 con 0x7fd268100270 2023-10-07T23:17:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.589+0000 7fd24f7fe700 1 -- 172.21.15.67:0/604734210 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5515 (secure 0 0 0) 0x7fd25c093890 con 0x7fd268100270 2023-10-07T23:17:10.589 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 91 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:fsid 227de690-6566-11ee-8db5-212e2dc638e7 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-10-07T23:07:33.436637+0000 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2023-10-07T23:16:28.996447+0000 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 27 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client luminous 2023-10-07T23:17:10.590 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release quincy 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 12 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6802/1635424780,v1:172.21.15.67:6803/1635424780] [v2:172.21.15.67:6804/1635424780,v1:172.21.15.67:6805/1635424780] exists,up a7d0d95f-5162-4fdd-8f39-a33aa62127a6 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/3463493653,v1:172.21.15.86:6801/3463493653] [v2:172.21.15.86:6802/3463493653,v1:172.21.15.86:6803/3463493653] exists,up 04b0f276-f836-435c-88ac-d6832505fc36 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/3773700997,v1:172.21.15.123:6801/3773700997] [v2:172.21.15.123:6802/3773700997,v1:172.21.15.123:6803/3773700997] exists,up 8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6810/2933401597,v1:172.21.15.67:6811/2933401597] [v2:172.21.15.67:6812/2933401597,v1:172.21.15.67:6813/2933401597] exists,up 0b364f2c-b808-4c14-b266-ae3dbe82c90a 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/4225579249,v1:172.21.15.86:6809/4225579249] [v2:172.21.15.86:6810/4225579249,v1:172.21.15.86:6811/4225579249] exists,up 75a0c533-a0cd-426d-9820-189b07ecef7b 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/476642923,v1:172.21.15.123:6809/476642923] [v2:172.21.15.123:6810/476642923,v1:172.21.15.123:6811/476642923] exists,up 1266769f-5b0e-44be-adc4-2175cdb40312 2023-10-07T23:17:10.591 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6818/3799339954,v1:172.21.15.67:6819/3799339954] [v2:172.21.15.67:6820/3799339954,v1:172.21.15.67:6821/3799339954] exists,up 6bb57d51-c3a7-473e-9258-8f1dfe6c2761 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/3974856027,v1:172.21.15.86:6817/3974856027] [v2:172.21.15.86:6818/3974856027,v1:172.21.15.86:6819/3974856027] exists,up ab847d18-8863-428b-9647-eadc654dc957 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/1197944476,v1:172.21.15.123:6817/1197944476] [v2:172.21.15.123:6818/1197944476,v1:172.21.15.123:6819/1197944476] exists,up e766f43b-63d1-4a5c-a427-2caca15c2af6 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6826/92905414,v1:172.21.15.67:6827/92905414] [v2:172.21.15.67:6828/92905414,v1:172.21.15.67:6829/92905414] exists,up 3781cb60-d57e-40f1-a762-5ddfbfa66a15 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/2965800150,v1:172.21.15.86:6825/2965800150] [v2:172.21.15.86:6826/2965800150,v1:172.21.15.86:6827/2965800150] exists,up c315d67a-7a94-4907-9201-c76d2459b5f7 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/2458295261,v1:172.21.15.123:6825/2458295261] [v2:172.21.15.123:6826/2458295261,v1:172.21.15.123:6827/2458295261] exists,up 6efe153c-3d19-4e94-b8f2-40f78688c379 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.0 [7,11] 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.6 [8,11] 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.b [2,4] 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.e [2,4] 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.1a [8,1] 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.24 [4,9] 2023-10-07T23:17:10.592 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.3 [5,8] 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.17 [9,8] 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.1c [9,8] 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.1f [9,10] 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2785096143 expires 2023-10-08T23:08:27.834910+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3552735690 expires 2023-10-08T23:08:27.834910+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/2928618291 expires 2023-10-08T23:08:27.834910+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3749023824 expires 2023-10-08T23:08:02.345521+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1125966089 expires 2023-10-08T23:08:02.345521+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/3941151123 expires 2023-10-08T23:08:02.345521+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/3941151123 expires 2023-10-08T23:08:02.345521+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1865957213 expires 2023-10-08T23:08:27.834910+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3750679498 expires 2023-10-08T23:08:02.345521+0000 2023-10-07T23:17:10.593 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/2928618291 expires 2023-10-08T23:08:27.834910+0000 2023-10-07T23:17:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd25009b770 msgr2=0x7fd25009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd25009b770 0x7fd25009dc20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd258004890 tx=0x7fd258009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 msgr2=0x7fd2681102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 0x7fd2681102e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00ebc0 tx=0x7fd25c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 shutdown_connections 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd25009b770 0x7fd25009dc20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd268100c00 0x7fd268117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2680ff060 0x7fd26810fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 --2- 172.21.15.67:0/604734210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd268100270 0x7fd2681102e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 >> 172.21.15.67:0/604734210 conn(0x7fd2680fa7c0 msgr2=0x7fd268103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.591+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 shutdown_connections 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:10.592+0000 7fd26d335700 1 -- 172.21.15.67:0/604734210 wait complete. 2023-10-07T23:17:11.008 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-07T23:17:11.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:17:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 -- 172.21.15.67:0/1328061179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fe880 msgr2=0x7f9f54105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1328061179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fe880 0x7f9f54105140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004ca0 tx=0x7f9f4c009ca0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 -- 172.21.15.67:0/1328061179 shutdown_connections 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1328061179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fe880 0x7f9f54105140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1328061179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f540fcce0 0x7f9f540fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1328061179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fdef0 0x7f9f540fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.217+0000 7f9f589fe700 1 -- 172.21.15.67:0/1328061179 >> 172.21.15.67:0/1328061179 conn(0x7f9f540f8460 msgr2=0x7f9f540fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.218+0000 7f9f589fe700 1 -- 172.21.15.67:0/1328061179 shutdown_connections 2023-10-07T23:17:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.218+0000 7f9f589fe700 1 -- 172.21.15.67:0/1328061179 wait complete. 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.218+0000 7f9f589fe700 1 Processor -- start 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 -- start start 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 0x7f9f5410f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fdef0 0x7f9f5410ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f540fe880 0x7f9f54116f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541175a0 con 0x7f9f540fe880 2023-10-07T23:17:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54117710 con 0x7f9f540fcce0 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f589fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541178d0 con 0x7f9f540fdef0 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f52d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 0x7f9f5410f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f52d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 0x7f9f5410f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:47210/0 (socket says 172.21.15.67:47210) 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f52d9d700 1 -- 172.21.15.67:0/1115357405 learned_addr learned my addr 172.21.15.67:0/1115357405 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f5359e700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f540fe880 0x7f9f54116f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.219+0000 7f9f5259c700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fdef0 0x7f9f5410ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f52d9d700 1 -- 172.21.15.67:0/1115357405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fdef0 msgr2=0x7f9f5410ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f52d9d700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fdef0 0x7f9f5410ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f52d9d700 1 -- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f540fe880 msgr2=0x7f9f54116f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f52d9d700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f540fe880 0x7f9f54116f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f52d9d700 1 -- 172.21.15.67:0/1115357405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44005040 con 0x7f9f540fcce0 2023-10-07T23:17:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f52d9d700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 0x7f9f5410f9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009730 tx=0x7f9f44009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44007be0 con 0x7f9f540fcce0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c0048f0 con 0x7f9f540fcce0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f4400f040 con 0x7f9f540fcce0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44013b80 con 0x7f9f540fcce0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.220+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f54117f80 con 0x7f9f540fcce0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.221+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f54108550 con 0x7f9f540fcce0 2023-10-07T23:17:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.223+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9f4400e840 con 0x7f9f540fcce0 2023-10-07T23:17:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.224+0000 7f9f3b7fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9f3c09b770 0x7f9f3c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.225+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f9f440c55f0 con 0x7f9f540fcce0 2023-10-07T23:17:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.225+0000 7f9f5259c700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9f3c09b770 0x7f9f3c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.225+0000 7f9f5259c700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9f3c09b770 0x7f9f3c09dc20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009b80 tx=0x7f9f4800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.227+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9f44090aa0 con 0x7f9f540fcce0 2023-10-07T23:17:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:11 smithi067 ceph-mon[82723]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:11.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:11 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/604734210' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-07T23:17:11.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:11 smithi123 ceph-mon[122926]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:11.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:11 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/604734210' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-07T23:17:11.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.430+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f54118110 con 0x7f9f540fcce0 2023-10-07T23:17:11.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.432+0000 7f9f3b7fe700 1 -- 172.21.15.67:0/1115357405 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7f9f440940c0 con 0x7f9f540fcce0 2023-10-07T23:17:11.433 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:11.433 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:17:11.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9f3c09b770 msgr2=0x7f9f3c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9f3c09b770 0x7f9f3c09dc20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009b80 tx=0x7f9f4800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:11.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 msgr2=0x7f9f5410f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 0x7f9f5410f9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009730 tx=0x7f9f44009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 shutdown_connections 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f540fcce0 0x7f9f5410f9c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9f3c09b770 0x7f9f3c09dc20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f540fe880 0x7f9f54116f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 --2- 172.21.15.67:0/1115357405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f540fdef0 0x7f9f5410ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 >> 172.21.15.67:0/1115357405 conn(0x7f9f540f8460 msgr2=0x7f9f540fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:11.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 shutdown_connections 2023-10-07T23:17:11.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.434+0000 7f9f589fe700 1 -- 172.21.15.67:0/1115357405 wait complete. 2023-10-07T23:17:11.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:11 smithi086 ceph-mon[122131]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:11.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:11 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/604734210' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-07T23:17:11.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:15:25.227955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:15:26.323230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-07T23:15:29.861861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:17:11.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:17:11.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 -- 172.21.15.67:0/1183987734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f00fd850 msgr2=0x7f40f00ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/1183987734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f00fd850 0x7f40f00ffc30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e40049c0 tx=0x7f40e4009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 -- 172.21.15.67:0/1183987734 shutdown_connections 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/1183987734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f0102a90 0x7f40f0104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/1183987734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f00fd850 0x7f40f00ffc30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/1183987734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0100170 0x7f40f0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 -- 172.21.15.67:0/1183987734 >> 172.21.15.67:0/1183987734 conn(0x7f40f00f7370 msgr2=0x7f40f00f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 -- 172.21.15.67:0/1183987734 shutdown_connections 2023-10-07T23:17:11.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.648+0000 7f40f5f9d700 1 -- 172.21.15.67:0/1183987734 wait complete. 2023-10-07T23:17:11.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 Processor -- start 2023-10-07T23:17:11.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 -- start start 2023-10-07T23:17:11.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f00fd850 0x7f40f00fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f0100170 0x7f40f00fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 0x7f40f00fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00fc410 con 0x7f40f0102a90 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00fc580 con 0x7f40f0100170 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f5f9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0117a60 con 0x7f40f00fd850 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f579c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 0x7f40f00fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40effff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f0100170 0x7f40f00fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f579c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 0x7f40f00fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60582/0 (socket says 172.21.15.67:60582) 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f579c700 1 -- 172.21.15.67:0/952846227 learned_addr learned my addr 172.21.15.67:0/952846227 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.649+0000 7f40f4f9b700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f00fd850 0x7f40f00fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f579c700 1 -- 172.21.15.67:0/952846227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f00fd850 msgr2=0x7f40f00fd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f579c700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f00fd850 0x7f40f00fd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f579c700 1 -- 172.21.15.67:0/952846227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f0100170 msgr2=0x7f40f00fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f579c700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f0100170 0x7f40f00fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f579c700 1 -- 172.21.15.67:0/952846227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e4005040 con 0x7f40f0102a90 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f579c700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 0x7f40f00fbca0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40e800ec20 tx=0x7f40e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8007b30 con 0x7f40f0102a90 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f0117cb0 con 0x7f40f0102a90 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e800f040 con 0x7f40f0102a90 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e800ca10 con 0x7f40f0102a90 2023-10-07T23:17:11.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.650+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f0118220 con 0x7f40f0102a90 2023-10-07T23:17:11.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.651+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f00f8ea0 con 0x7f40f0102a90 2023-10-07T23:17:11.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.653+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f40e8007c90 con 0x7f40f0102a90 2023-10-07T23:17:11.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.656+0000 7f40ed7fa700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f40e009b7f0 0x7f40e009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:11.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.656+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f40e80c5660 con 0x7f40f0102a90 2023-10-07T23:17:11.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.656+0000 7f40f4f9b700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f40e009b7f0 0x7f40e009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:11.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.656+0000 7f40f4f9b700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f40e009b7f0 0x7f40e009dca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009b30 tx=0x7f40e4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:11.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.660+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f40e8090b10 con 0x7f40f0102a90 2023-10-07T23:17:11.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.864+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40f002b370 con 0x7f40f0102a90 2023-10-07T23:17:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.864+0000 7f40ed7fa700 1 -- 172.21.15.67:0/952846227 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40e8094130 con 0x7f40f0102a90 2023-10-07T23:17:11.865 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:17:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.866+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f40e009b7f0 msgr2=0x7f40e009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.866+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f40e009b7f0 0x7f40e009dca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009b30 tx=0x7f40e4009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.866+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 msgr2=0x7f40f00fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.866+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 0x7f40f00fbca0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40e800ec20 tx=0x7f40e800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 shutdown_connections 2023-10-07T23:17:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f40e009b7f0 0x7f40e009dca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f0102a90 0x7f40f00fbca0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f00fd850 0x7f40f00fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 --2- 172.21.15.67:0/952846227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f0100170 0x7f40f00fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 >> 172.21.15.67:0/952846227 conn(0x7f40f00f7370 msgr2=0x7f40f00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 shutdown_connections 2023-10-07T23:17:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:11.867+0000 7f40f5f9d700 1 -- 172.21.15.67:0/952846227 wait complete. 2023-10-07T23:17:11.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:17:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/36973838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8070570 msgr2=0x7f3bb80709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/36973838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8070570 0x7f3bb80709c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0049c0 tx=0x7f3bac009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/36973838 shutdown_connections 2023-10-07T23:17:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/36973838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8104b90 0x7f3bb8107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/36973838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb8071bc0 0x7f3bb8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/36973838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8070570 0x7f3bb80709c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/36973838 >> 172.21.15.67:0/36973838 conn(0x7f3bb80f8450 msgr2=0x7f3bb80fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/36973838 shutdown_connections 2023-10-07T23:17:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/36973838 wait complete. 2023-10-07T23:17:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.080+0000 7f3bbcd9d700 1 Processor -- start 2023-10-07T23:17:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 -- start start 2023-10-07T23:17:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb8070570 0x7f3bb810fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 0x7f3bb81102e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8104b90 0x7f3bb8117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8117970 con 0x7f3bb8071bc0 2023-10-07T23:17:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8117ae0 con 0x7f3bb8104b90 2023-10-07T23:17:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bbcd9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8117ca0 con 0x7f3bb8070570 2023-10-07T23:17:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bb6ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 0x7f3bb81102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bb6ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 0x7f3bb81102e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37892/0 (socket says 172.21.15.67:37892) 2023-10-07T23:17:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bb6ffd700 1 -- 172.21.15.67:0/2188654438 learned_addr learned my addr 172.21.15.67:0/2188654438 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bb77fe700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb8070570 0x7f3bb810fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.081+0000 7f3bb7fff700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8104b90 0x7f3bb8117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bb6ffd700 1 -- 172.21.15.67:0/2188654438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb8070570 msgr2=0x7f3bb810fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bb6ffd700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb8070570 0x7f3bb810fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bb6ffd700 1 -- 172.21.15.67:0/2188654438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8104b90 msgr2=0x7f3bb8117330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bb6ffd700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8104b90 0x7f3bb8117330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bb6ffd700 1 -- 172.21.15.67:0/2188654438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac005040 con 0x7f3bb8071bc0 2023-10-07T23:17:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bb6ffd700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 0x7f3bb81102e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009180 tx=0x7f3bac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac020070 con 0x7f3bb8071bc0 2023-10-07T23:17:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bac009520 con 0x7f3bb8071bc0 2023-10-07T23:17:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb8117f20 con 0x7f3bb8071bc0 2023-10-07T23:17:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00e070 con 0x7f3bb8071bc0 2023-10-07T23:17:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.082+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb81182c0 con 0x7f3bb8071bc0 2023-10-07T23:17:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.083+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb810a740 con 0x7f3bb8071bc0 2023-10-07T23:17:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.085+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f3bac009690 con 0x7f3bb8071bc0 2023-10-07T23:17:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.086+0000 7f3b9ffff700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3ba009b770 0x7f3ba009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.087+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f3bac0c6c60 con 0x7f3bb8071bc0 2023-10-07T23:17:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.087+0000 7f3bb77fe700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3ba009b770 0x7f3ba009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.087+0000 7f3bb77fe700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3ba009b770 0x7f3ba009dc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8111290 tx=0x7f3ba800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.088+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f3bac092110 con 0x7f3bb8071bc0 2023-10-07T23:17:12.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:12 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1115357405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:12.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:12 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/952846227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:12.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.290+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bb8118570 con 0x7f3bb8071bc0 2023-10-07T23:17:12.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.291+0000 7f3b9ffff700 1 -- 172.21.15.67:0/2188654438 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bac095730 con 0x7f3bb8071bc0 2023-10-07T23:17:12.291 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3ba009b770 msgr2=0x7f3ba009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3ba009b770 0x7f3ba009dc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8111290 tx=0x7f3ba800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 msgr2=0x7f3bb81102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 0x7f3bb81102e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009180 tx=0x7f3bac009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 shutdown_connections 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bb8104b90 0x7f3bb8117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3ba009b770 0x7f3ba009dc20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb8070570 0x7f3bb810fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 --2- 172.21.15.67:0/2188654438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb8071bc0 0x7f3bb81102e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 >> 172.21.15.67:0/2188654438 conn(0x7f3bb80f8450 msgr2=0x7f3bb80fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 shutdown_connections 2023-10-07T23:17:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.293+0000 7f3bbcd9d700 1 -- 172.21.15.67:0/2188654438 wait complete. 2023-10-07T23:17:12.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:17:12.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:12 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1115357405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:12.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:12 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/952846227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:12.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:12 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1115357405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:12.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:12 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/952846227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 -- 172.21.15.67:0/2001747945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c100170 msgr2=0x7fe86c102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 --2- 172.21.15.67:0/2001747945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c100170 0x7fe86c102550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe85c004900 tx=0x7fe85c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 -- 172.21.15.67:0/2001747945 shutdown_connections 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 --2- 172.21.15.67:0/2001747945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c102a90 0x7fe86c104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 --2- 172.21.15.67:0/2001747945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c0fd850 0x7fe86c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 --2- 172.21.15.67:0/2001747945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c100170 0x7fe86c102550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 -- 172.21.15.67:0/2001747945 >> 172.21.15.67:0/2001747945 conn(0x7fe86c0f7370 msgr2=0x7fe86c0f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:12.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 -- 172.21.15.67:0/2001747945 shutdown_connections 2023-10-07T23:17:12.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.503+0000 7fe87359d700 1 -- 172.21.15.67:0/2001747945 wait complete. 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 Processor -- start 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 -- start start 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 0x7fe86c0fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c100170 0x7fe86c0fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c102a90 0x7fe86c0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0fc410 con 0x7fe86c102a90 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0fc580 con 0x7fe86c0fd850 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.504+0000 7fe87359d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c117b20 con 0x7fe86c100170 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe87259b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 0x7fe86c0fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe87259b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 0x7fe86c0fd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39520/0 (socket says 172.21.15.67:39520) 2023-10-07T23:17:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe87259b700 1 -- 172.21.15.67:0/3009351985 learned_addr learned my addr 172.21.15.67:0/3009351985 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe872d9c700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c102a90 0x7fe86c0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c100170 0x7fe86c0fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 -- 172.21.15.67:0/3009351985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 msgr2=0x7fe86c0fd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 0x7fe86c0fd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 -- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c102a90 msgr2=0x7fe86c0fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c102a90 0x7fe86c0fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 -- 172.21.15.67:0/3009351985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c005040 con 0x7fe86c100170 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe872d9c700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c102a90 0x7fe86c0fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe87259b700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 0x7fe86c0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.505+0000 7fe871d9a700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c100170 0x7fe86c0fb760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0047d0 tx=0x7fe85c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.506+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00e040 con 0x7fe86c100170 2023-10-07T23:17:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.506+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe85c0097c0 con 0x7fe86c100170 2023-10-07T23:17:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.506+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c117d70 con 0x7fe86c100170 2023-10-07T23:17:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.506+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c01cd80 con 0x7fe86c100170 2023-10-07T23:17:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.506+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c118200 con 0x7fe86c100170 2023-10-07T23:17:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.507+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c0f8ea0 con 0x7fe86c100170 2023-10-07T23:17:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.509+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fe85c01c430 con 0x7fe86c100170 2023-10-07T23:17:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.510+0000 7fe862ffd700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe85809b7f0 0x7fe85809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.510+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fe85c0c6ab0 con 0x7fe86c100170 2023-10-07T23:17:12.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.510+0000 7fe87259b700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe85809b7f0 0x7fe85809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.511+0000 7fe87259b700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe85809b7f0 0x7fe85809dca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe868009b80 tx=0x7fe86800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:12.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.512+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fe85c092040 con 0x7fe86c100170 2023-10-07T23:17:12.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.715+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe86c108570 con 0x7fe86c100170 2023-10-07T23:17:12.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.715+0000 7fe862ffd700 1 -- 172.21.15.67:0/3009351985 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe85c095660 con 0x7fe86c100170 2023-10-07T23:17:12.716 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-07T23:17:12.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe85809b7f0 msgr2=0x7fe85809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe85809b7f0 0x7fe85809dca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe868009b80 tx=0x7fe86800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c100170 msgr2=0x7fe86c0fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c100170 0x7fe86c0fb760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0047d0 tx=0x7fe85c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 shutdown_connections 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe86c0fd850 0x7fe86c0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe85809b7f0 0x7fe85809dca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c102a90 0x7fe86c0fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 --2- 172.21.15.67:0/3009351985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86c100170 0x7fe86c0fb760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 >> 172.21.15.67:0/3009351985 conn(0x7fe86c0f7370 msgr2=0x7fe86c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:12.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 shutdown_connections 2023-10-07T23:17:12.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.718+0000 7fe87359d700 1 -- 172.21.15.67:0/3009351985 wait complete. 2023-10-07T23:17:12.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.927+0000 7fb7e319d700 1 -- 172.21.15.67:0/640328054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fdf00 msgr2=0x7fb7dc0fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.927+0000 7fb7e319d700 1 --2- 172.21.15.67:0/640328054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc0fe350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0049c0 tx=0x7fb7cc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 -- 172.21.15.67:0/640328054 shutdown_connections 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 --2- 172.21.15.67:0/640328054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 0x7fb7dc105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 --2- 172.21.15.67:0/640328054 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fccf0 0x7fb7dc0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 --2- 172.21.15.67:0/640328054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc0fe350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 -- 172.21.15.67:0/640328054 >> 172.21.15.67:0/640328054 conn(0x7fb7dc0f8410 msgr2=0x7fb7dc0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 -- 172.21.15.67:0/640328054 shutdown_connections 2023-10-07T23:17:12.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 -- 172.21.15.67:0/640328054 wait complete. 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.928+0000 7fb7e319d700 1 Processor -- start 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 -- start start 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fccf0 0x7fb7dc10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 0x7fb7dc114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc1155c0 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc115730 con 0x7fb7dc0fe890 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e319d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc1158f0 con 0x7fb7dc0fccf0 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e199a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e199a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc10df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37926/0 (socket says 172.21.15.67:37926) 2023-10-07T23:17:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e199a700 1 -- 172.21.15.67:0/2189481354 learned_addr learned my addr 172.21.15.67:0/2189481354 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e299c700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 0x7fb7dc114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e219b700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fccf0 0x7fb7dc10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e199a700 1 -- 172.21.15.67:0/2189481354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fccf0 msgr2=0x7fb7dc10d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e199a700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fccf0 0x7fb7dc10d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.929+0000 7fb7e199a700 1 -- 172.21.15.67:0/2189481354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 msgr2=0x7fb7dc114fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7e199a700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 0x7fb7dc114fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7e199a700 1 -- 172.21.15.67:0/2189481354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8005040 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7e299c700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 0x7fb7dc114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7e199a700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc10df20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009180 tx=0x7fb7cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc020070 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7cc005040 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7cc009520 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00e070 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.930+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc115ec0 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.931+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc103c00 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.934+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb7cc009690 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.936+0000 7fb7d2ffd700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb7c809b770 0x7fb7c809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:12.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.936+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb7cc0c6c10 con 0x7fb7dc0fdf00 2023-10-07T23:17:12.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.936+0000 7fb7e219b700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb7c809b770 0x7fb7c809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:12.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.937+0000 7fb7e219b700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb7c809b770 0x7fb7c809dc20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0fdd60 tx=0x7fb7d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:12.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:12.938+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fb7cc0920c0 con 0x7fb7dc0fdf00 2023-10-07T23:17:13.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.141+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb7dc108530 con 0x7fb7dc0fdf00 2023-10-07T23:17:13.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.142+0000 7fb7d2ffd700 1 -- 172.21.15.67:0/2189481354 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb7cc0956e0 con 0x7fb7dc0fdf00 2023-10-07T23:17:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:13.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb7c809b770 msgr2=0x7fb7c809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb7c809b770 0x7fb7c809dc20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0fdd60 tx=0x7fb7d800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 msgr2=0x7fb7dc10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc10df20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009180 tx=0x7fb7cc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 shutdown_connections 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7dc0fe890 0x7fb7dc114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb7c809b770 0x7fb7c809dc20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7dc0fccf0 0x7fb7dc10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 --2- 172.21.15.67:0/2189481354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7dc0fdf00 0x7fb7dc10df20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 >> 172.21.15.67:0/2189481354 conn(0x7fb7dc0f8410 msgr2=0x7fb7dc0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 shutdown_connections 2023-10-07T23:17:13.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.144+0000 7fb7e319d700 1 -- 172.21.15.67:0/2189481354 wait complete. 2023-10-07T23:17:13.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:17:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 -- 172.21.15.67:0/3187847179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 msgr2=0x7f15700fe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 --2- 172.21.15.67:0/3187847179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f15700fe360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15600049c0 tx=0x7f1560009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 -- 172.21.15.67:0/3187847179 shutdown_connections 2023-10-07T23:17:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 --2- 172.21.15.67:0/3187847179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15700fcd00 0x7f15700fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 --2- 172.21.15.67:0/3187847179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15700fe8a0 0x7f1570105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 --2- 172.21.15.67:0/3187847179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f15700fe360 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 -- 172.21.15.67:0/3187847179 >> 172.21.15.67:0/3187847179 conn(0x7f15700f8420 msgr2=0x7f15700fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 -- 172.21.15.67:0/3187847179 shutdown_connections 2023-10-07T23:17:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.351+0000 7f157659d700 1 -- 172.21.15.67:0/3187847179 wait complete. 2023-10-07T23:17:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.352+0000 7f157659d700 1 Processor -- start 2023-10-07T23:17:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.352+0000 7f157659d700 1 -- start start 2023-10-07T23:17:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157659d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15700fcd00 0x7f157010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157659d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f157010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157659d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15700fe8a0 0x7f1570114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157659d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570115510 con 0x7f15700fdf10 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157659d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570115680 con 0x7f15700fe8a0 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157659d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570115840 con 0x7f15700fcd00 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f157010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f157010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37934/0 (socket says 172.21.15.67:37934) 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 -- 172.21.15.67:0/2760831003 learned_addr learned my addr 172.21.15.67:0/2760831003 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f157559b700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15700fcd00 0x7f157010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1575d9c700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15700fe8a0 0x7f1570114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 -- 172.21.15.67:0/2760831003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15700fcd00 msgr2=0x7f157010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15700fcd00 0x7f157010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 -- 172.21.15.67:0/2760831003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15700fe8a0 msgr2=0x7f1570114f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15700fe8a0 0x7f1570114f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.353+0000 7f1574d9a700 1 -- 172.21.15.67:0/2760831003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c005040 con 0x7f15700fdf10 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.354+0000 7f1574d9a700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f157010de70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1560013830 tx=0x7f1560009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.354+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000e050 con 0x7f15700fdf10 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.354+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156001b3f0 con 0x7f15700fdf10 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.354+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1560005040 con 0x7f15700fdf10 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.354+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560012630 con 0x7f15700fdf10 2023-10-07T23:17:13.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.354+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1570115e60 con 0x7f15700fdf10 2023-10-07T23:17:13.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.355+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15701041d0 con 0x7f15700fdf10 2023-10-07T23:17:13.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.357+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f156001b560 con 0x7f15700fdf10 2023-10-07T23:17:13.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.358+0000 7f1565ffb700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f155c09b7f0 0x7f155c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.358+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f15600c63c0 con 0x7f15700fdf10 2023-10-07T23:17:13.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.358+0000 7f157559b700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f155c09b7f0 0x7f155c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.359+0000 7f157559b700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f155c09b7f0 0x7f155c09dca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f15700fdd70 tx=0x7f156c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:13.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.362+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f15600918d0 con 0x7f15700fdf10 2023-10-07T23:17:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:13 smithi123 ceph-mon[122926]: pgmap v343: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:13.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:13 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2188654438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:13.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:13 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3009351985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:13.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:13 smithi086 ceph-mon[122131]: pgmap v343: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:13.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:13 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2188654438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:13.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:13 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3009351985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:13.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:13 smithi067 ceph-mon[82723]: pgmap v343: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:13.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:13 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2188654438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:13.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:13 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3009351985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:13.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.615+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1570115a00 con 0x7f15700fdf10 2023-10-07T23:17:13.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.616+0000 7f1565ffb700 1 -- 172.21.15.67:0/2760831003 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f1560028090 con 0x7f15700fdf10 2023-10-07T23:17:13.617 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:13.617 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:17:13.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.618+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f155c09b7f0 msgr2=0x7f155c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.618+0000 7f157659d700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f155c09b7f0 0x7f155c09dca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f15700fdd70 tx=0x7f156c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:13.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.618+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 msgr2=0x7f157010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.618+0000 7f157659d700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f157010de70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1560013830 tx=0x7f1560009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.618+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 shutdown_connections 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15700fe8a0 0x7f1570114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f155c09b7f0 0x7f155c09dca0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15700fcd00 0x7f157010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 --2- 172.21.15.67:0/2760831003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700fdf10 0x7f157010de70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 >> 172.21.15.67:0/2760831003 conn(0x7f15700f8420 msgr2=0x7f1570101350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:13.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 shutdown_connections 2023-10-07T23:17:13.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.619+0000 7f157659d700 1 -- 172.21.15.67:0/2760831003 wait complete. 2023-10-07T23:17:13.621 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:17:13.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 -- 172.21.15.67:0/473718767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fdef0 msgr2=0x7f34b00fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/473718767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fdef0 0x7f34b00fe340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f34a00049c0 tx=0x7f34a0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 -- 172.21.15.67:0/473718767 shutdown_connections 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/473718767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b00fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/473718767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fe880 0x7f34b0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/473718767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fdef0 0x7f34b00fe340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 -- 172.21.15.67:0/473718767 >> 172.21.15.67:0/473718767 conn(0x7f34b00f8460 msgr2=0x7f34b00fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 -- 172.21.15.67:0/473718767 shutdown_connections 2023-10-07T23:17:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.828+0000 7f34b6d9d700 1 -- 172.21.15.67:0/473718767 wait complete. 2023-10-07T23:17:13.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 Processor -- start 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 -- start start 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b010fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 0x7f34b0110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fe880 0x7f34b01171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b01177e0 con 0x7f34b00fe880 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b0117950 con 0x7f34b00fcce0 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b6d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b0117b10 con 0x7f34b00fdef0 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.829+0000 7f34b659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fe880 0x7f34b01171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b559a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 0x7f34b0110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b010fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b010fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39590/0 (socket says 172.21.15.67:39590) 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 -- 172.21.15.67:0/1587751920 learned_addr learned my addr 172.21.15.67:0/1587751920 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fe880 0x7f34b01171a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37944/0 (socket says 172.21.15.67:37944) 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b559a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 0x7f34b0110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:51218/0 (socket says 172.21.15.67:51218) 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 -- 172.21.15.67:0/1587751920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 msgr2=0x7f34b0110150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 0x7f34b0110150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 -- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fe880 msgr2=0x7f34b01171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fe880 0x7f34b01171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 -- 172.21.15.67:0/1587751920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac005040 con 0x7f34b00fcce0 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b559a700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 0x7f34b0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.830+0000 7f34b5d9b700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b010fc10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009750 tx=0x7f34ac009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:13.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.831+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac007c80 con 0x7f34b00fcce0 2023-10-07T23:17:13.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.831+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a0005040 con 0x7f34b00fcce0 2023-10-07T23:17:13.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.831+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ac00f040 con 0x7f34b00fcce0 2023-10-07T23:17:13.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.831+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac013c20 con 0x7f34b00fcce0 2023-10-07T23:17:13.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.831+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b0118140 con 0x7f34b00fcce0 2023-10-07T23:17:13.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.832+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b0108550 con 0x7f34b00fcce0 2023-10-07T23:17:13.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.834+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f34ac00e7e0 con 0x7f34b00fcce0 2023-10-07T23:17:13.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.835+0000 7f34a67fc700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f349c09b770 0x7f349c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:13.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.835+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f34ac0c5d30 con 0x7f34b00fcce0 2023-10-07T23:17:13.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.835+0000 7f34b559a700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f349c09b770 0x7f349c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:13.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.836+0000 7f34b559a700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f349c09b770 0x7f349c09dc20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34a0004890 tx=0x7f34a0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:13.837+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f34ac0911e0 con 0x7f34b00fcce0 2023-10-07T23:17:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.044+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34b01183c0 con 0x7f34b00fcce0 2023-10-07T23:17:14.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.046+0000 7f34a67fc700 1 -- 172.21.15.67:0/1587751920 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7f34ac094800 con 0x7f34b00fcce0 2023-10-07T23:17:14.047 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:14.047 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:17:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f349c09b770 msgr2=0x7f349c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f349c09b770 0x7f349c09dc20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34a0004890 tx=0x7f34a0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 msgr2=0x7f34b010fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b010fc10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009750 tx=0x7f34ac009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 shutdown_connections 2023-10-07T23:17:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34b00fcce0 0x7f34b010fc10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f349c09b770 0x7f349c09dc20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b00fe880 0x7f34b01171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 --2- 172.21.15.67:0/1587751920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b00fdef0 0x7f34b0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 >> 172.21.15.67:0/1587751920 conn(0x7f34b00f8460 msgr2=0x7f34b00fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 shutdown_connections 2023-10-07T23:17:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.048+0000 7f34b6d9d700 1 -- 172.21.15.67:0/1587751920 wait complete. 2023-10-07T23:17:14.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:15:25.227955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:15:26.323230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-07T23:15:29.861861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:17:14.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:17:14.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 -- 172.21.15.67:0/4150804487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fccf0 msgr2=0x7f29a40fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/4150804487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fccf0 0x7f29a40fd100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29940049c0 tx=0x7f2994009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 -- 172.21.15.67:0/4150804487 shutdown_connections 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/4150804487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fe890 0x7f29a4105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/4150804487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fccf0 0x7f29a40fd100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/4150804487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a40fdf00 0x7f29a40fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 -- 172.21.15.67:0/4150804487 >> 172.21.15.67:0/4150804487 conn(0x7f29a40f8450 msgr2=0x7f29a40fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 -- 172.21.15.67:0/4150804487 shutdown_connections 2023-10-07T23:17:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.258+0000 7f29a8d9d700 1 -- 172.21.15.67:0/4150804487 wait complete. 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 Processor -- start 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 -- start start 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fccf0 0x7f29a410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a40fdf00 0x7f29a410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 0x7f29a4114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4115580 con 0x7f29a40fe890 2023-10-07T23:17:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a41156f0 con 0x7f29a40fccf0 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a8d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a41158b0 con 0x7f29a40fdf00 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 0x7f29a4114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a37fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fccf0 0x7f29a410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 0x7f29a4114f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37958/0 (socket says 172.21.15.67:37958) 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a3fff700 1 -- 172.21.15.67:0/2823701284 learned_addr learned my addr 172.21.15.67:0/2823701284 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.259+0000 7f29a2ffd700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a40fdf00 0x7f29a410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a37fe700 1 -- 172.21.15.67:0/2823701284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a40fdf00 msgr2=0x7f29a410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a37fe700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a40fdf00 0x7f29a410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a37fe700 1 -- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 msgr2=0x7f29a4114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a37fe700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 0x7f29a4114f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a37fe700 1 -- 172.21.15.67:0/2823701284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994005040 con 0x7f29a40fccf0 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a3fff700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 0x7f29a4114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f29a37fe700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fccf0 0x7f29a410d9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2994009fd0 tx=0x7f2994009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299400e040 con 0x7f29a40fccf0 2023-10-07T23:17:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.260+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2994009670 con 0x7f29a40fccf0 2023-10-07T23:17:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.261+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a4115b30 con 0x7f29a40fccf0 2023-10-07T23:17:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.261+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994013d30 con 0x7f29a40fccf0 2023-10-07T23:17:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.261+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a4115f90 con 0x7f29a40fccf0 2023-10-07T23:17:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.262+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a4103b80 con 0x7f29a40fccf0 2023-10-07T23:17:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.264+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f2994013420 con 0x7f29a40fccf0 2023-10-07T23:17:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.265+0000 7f298bfff700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f298c09b770 0x7f298c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.265+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f29940c6970 con 0x7f29a40fccf0 2023-10-07T23:17:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.265+0000 7f29a2ffd700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f298c09b770 0x7f298c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.266+0000 7f29a2ffd700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f298c09b770 0x7f298c09dc20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2998004f60 tx=0x7f299800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.267+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2994091e20 con 0x7f29a40fccf0 2023-10-07T23:17:14.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:14 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2189481354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:14.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:14 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2760831003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:14.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:14 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2189481354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:14.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:14 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2760831003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:14.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.471+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29a402b320 con 0x7f29a40fccf0 2023-10-07T23:17:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.472+0000 7f298bfff700 1 -- 172.21.15.67:0/2823701284 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2994095440 con 0x7f29a40fccf0 2023-10-07T23:17:14.472 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:17:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f298c09b770 msgr2=0x7f298c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f298c09b770 0x7f298c09dc20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2998004f60 tx=0x7f299800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fccf0 msgr2=0x7f29a410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fccf0 0x7f29a410d9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2994009fd0 tx=0x7f2994009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 shutdown_connections 2023-10-07T23:17:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a40fccf0 0x7f29a410d9a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f298c09b770 0x7f298c09dc20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a40fe890 0x7f29a4114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 --2- 172.21.15.67:0/2823701284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a40fdf00 0x7f29a410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.474+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 >> 172.21.15.67:0/2823701284 conn(0x7f29a40f8450 msgr2=0x7f29a40fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.475+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 shutdown_connections 2023-10-07T23:17:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.475+0000 7f29a8d9d700 1 -- 172.21.15.67:0/2823701284 wait complete. 2023-10-07T23:17:14.490 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:14 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2189481354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:14.491 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:14 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2760831003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:14.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:17:14.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.683+0000 7f7bb7fff700 1 -- 172.21.15.67:0/712813101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fe890 msgr2=0x7f7bb8105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.683+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/712813101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fe890 0x7f7bb8105150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009d60 tx=0x7f7bb0004d10 comp rx=0 tx=0).stop 2023-10-07T23:17:14.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 -- 172.21.15.67:0/712813101 shutdown_connections 2023-10-07T23:17:14.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/712813101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80fccf0 0x7f7bb80fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/712813101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fe890 0x7f7bb8105150 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/712813101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bb80fdf00 0x7f7bb80fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.686 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 -- 172.21.15.67:0/712813101 >> 172.21.15.67:0/712813101 conn(0x7f7bb80f8450 msgr2=0x7f7bb80fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:14.686 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 -- 172.21.15.67:0/712813101 shutdown_connections 2023-10-07T23:17:14.686 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 -- 172.21.15.67:0/712813101 wait complete. 2023-10-07T23:17:14.686 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.684+0000 7f7bb7fff700 1 Processor -- start 2023-10-07T23:17:14.686 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 -- start start 2023-10-07T23:17:14.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bb80fccf0 0x7f7bb810f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 0x7f7bb810ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80fe890 0x7f7bb8116fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81175c0 con 0x7f7bb80fdf00 2023-10-07T23:17:14.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8117730 con 0x7f7bb80fe890 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb7fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81178f0 con 0x7f7bb80fccf0 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb67fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 0x7f7bb810ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb67fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 0x7f7bb810ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37974/0 (socket says 172.21.15.67:37974) 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb67fc700 1 -- 172.21.15.67:0/2866321840 learned_addr learned my addr 172.21.15.67:0/2866321840 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bb80fccf0 0x7f7bb810f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb77fe700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80fe890 0x7f7bb8116fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb67fc700 1 -- 172.21.15.67:0/2866321840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bb80fccf0 msgr2=0x7f7bb810f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.685+0000 7f7bb67fc700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bb80fccf0 0x7f7bb810f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7bb67fc700 1 -- 172.21.15.67:0/2866321840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80fe890 msgr2=0x7f7bb8116fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7bb67fc700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80fe890 0x7f7bb8116fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7bb67fc700 1 -- 172.21.15.67:0/2866321840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0009940 con 0x7f7bb80fdf00 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7bb67fc700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 0x7f7bb810ff20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00ebf0 tx=0x7f7bac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0047d0 con 0x7f7bb80fdf00 2023-10-07T23:17:14.690 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bac00f040 con 0x7f7bb80fdf00 2023-10-07T23:17:14.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb8117b70 con 0x7f7bb80fdf00 2023-10-07T23:17:14.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00c980 con 0x7f7bb80fdf00 2023-10-07T23:17:14.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.686+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb8118010 con 0x7f7bb80fdf00 2023-10-07T23:17:14.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.687+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb8108560 con 0x7f7bb80fdf00 2023-10-07T23:17:14.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.689+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f7bac0090d0 con 0x7f7bb80fdf00 2023-10-07T23:17:14.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.690+0000 7f7b9f7fe700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7ba009b770 0x7f7ba009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:14.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.690+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f7bac0c4e70 con 0x7f7bb80fdf00 2023-10-07T23:17:14.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.690+0000 7f7bb6ffd700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7ba009b770 0x7f7ba009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:14.693 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.691+0000 7f7bb6ffd700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7ba009b770 0x7f7ba009dc20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009b80 tx=0x7f7ba800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:14.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.694+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7bac090320 con 0x7f7bb80fdf00 2023-10-07T23:17:14.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.897+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7bb81186c0 con 0x7f7bb80fdf00 2023-10-07T23:17:14.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.898+0000 7f7b9f7fe700 1 -- 172.21.15.67:0/2866321840 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7bac0190a0 con 0x7f7bb80fdf00 2023-10-07T23:17:14.899 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7ba009b770 msgr2=0x7f7ba009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7ba009b770 0x7f7ba009dc20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009b80 tx=0x7f7ba800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 msgr2=0x7f7bb810ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 0x7f7bb810ff20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00ebf0 tx=0x7f7bac00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 shutdown_connections 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80fe890 0x7f7bb8116fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f7ba009b770 0x7f7ba009dc20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bb80fccf0 0x7f7bb810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 --2- 172.21.15.67:0/2866321840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bb80fdf00 0x7f7bb810ff20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:14.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 >> 172.21.15.67:0/2866321840 conn(0x7f7bb80f8450 msgr2=0x7f7bb806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:14.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 shutdown_connections 2023-10-07T23:17:14.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:14.900+0000 7f7bb7fff700 1 -- 172.21.15.67:0/2866321840 wait complete. 2023-10-07T23:17:14.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 -- 172.21.15.67:0/3036485641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 msgr2=0x7ffa9c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3036485641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c1045e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffa900049c0 tx=0x7ffa90009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 -- 172.21.15.67:0/3036485641 shutdown_connections 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3036485641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c066d10 0x7ffa9c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3036485641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c104bb0 0x7ffa9c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3036485641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c1045e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 -- 172.21.15.67:0/3036485641 >> 172.21.15.67:0/3036485641 conn(0x7ffa9c0f7370 msgr2=0x7ffa9c0f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 -- 172.21.15.67:0/3036485641 shutdown_connections 2023-10-07T23:17:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.110+0000 7ffaa119d700 1 -- 172.21.15.67:0/3036485641 wait complete. 2023-10-07T23:17:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 Processor -- start 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 -- start start 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c066d10 0x7ffa9c10fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c110350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c104bb0 0x7ffa9c1173e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c1179f0 con 0x7ffa9c102200 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c117b60 con 0x7ffa9c104bb0 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.111+0000 7ffaa119d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c117d20 con 0x7ffa9c066d10 2023-10-07T23:17:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c110350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c066d10 0x7ffa9c10fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c110350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37994/0 (socket says 172.21.15.67:37994) 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c066d10 0x7ffa9c10fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:51262/0 (socket says 172.21.15.67:51262) 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 -- 172.21.15.67:0/3300360731 learned_addr learned my addr 172.21.15.67:0/3300360731 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffaa099c700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c104bb0 0x7ffa9c1173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 -- 172.21.15.67:0/3300360731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c066d10 msgr2=0x7ffa9c10fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c066d10 0x7ffa9c10fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 -- 172.21.15.67:0/3300360731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c104bb0 msgr2=0x7ffa9c1173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c104bb0 0x7ffa9c1173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 -- 172.21.15.67:0/3300360731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa90005040 con 0x7ffa9c102200 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa9b7fe700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c110350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009400 tx=0x7ffa90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.112+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90020070 con 0x7ffa9c102200 2023-10-07T23:17:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.113+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c117fa0 con 0x7ffa9c102200 2023-10-07T23:17:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.113+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa900041a0 con 0x7ffa9c102200 2023-10-07T23:17:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.113+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000e070 con 0x7ffa9c102200 2023-10-07T23:17:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.113+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c1183a0 con 0x7ffa9c102200 2023-10-07T23:17:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.114+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c0f8ea0 con 0x7ffa9c102200 2023-10-07T23:17:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.116+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7ffa9001c6c0 con 0x7ffa9c102200 2023-10-07T23:17:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.117+0000 7ffa98ff9700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffa8409b770 0x7ffa8409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.117+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7ffa900c63a0 con 0x7ffa9c102200 2023-10-07T23:17:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.117+0000 7ffa9bfff700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffa8409b770 0x7ffa8409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.118+0000 7ffa9bfff700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffa8409b770 0x7ffa8409dc20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009b80 tx=0x7ffa8c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.121+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7ffa90091850 con 0x7ffa9c102200 2023-10-07T23:17:15.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:15 smithi067 ceph-mon[82723]: pgmap v344: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:17:15.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:15 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1587751920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:15.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:15 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2823701284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:15.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:15 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2866321840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:15.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.321+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa9c02b1e0 con 0x7ffa9c102200 2023-10-07T23:17:15.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.322+0000 7ffa98ff9700 1 -- 172.21.15.67:0/3300360731 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa90094e70 con 0x7ffa9c102200 2023-10-07T23:17:15.322 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffa8409b770 msgr2=0x7ffa8409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffa8409b770 0x7ffa8409dc20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009b80 tx=0x7ffa8c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 msgr2=0x7ffa9c110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c110350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009400 tx=0x7ffa90009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 shutdown_connections 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa9c104bb0 0x7ffa9c1173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7ffa8409b770 0x7ffa8409dc20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa9c066d10 0x7ffa9c10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 --2- 172.21.15.67:0/3300360731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa9c102200 0x7ffa9c110350 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 >> 172.21.15.67:0/3300360731 conn(0x7ffa9c0f7370 msgr2=0x7ffa9c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 shutdown_connections 2023-10-07T23:17:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.324+0000 7ffaa119d700 1 -- 172.21.15.67:0/3300360731 wait complete. 2023-10-07T23:17:15.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-07T23:17:15.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:15 smithi123 ceph-mon[122926]: pgmap v344: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:17:15.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:15 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1587751920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:15.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:15 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2823701284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:15.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:15 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2866321840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:15.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:15 smithi086 ceph-mon[122131]: pgmap v344: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-10-07T23:17:15.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:15 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1587751920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:15.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:15 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2823701284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:15.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:15 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2866321840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.537+0000 7f4fb999d700 1 -- 172.21.15.67:0/1454356257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4070570 msgr2=0x7f4fb40709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.537+0000 7f4fb999d700 1 --2- 172.21.15.67:0/1454356257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4070570 0x7f4fb40709c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00049c0 tx=0x7f4fa0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 -- 172.21.15.67:0/1454356257 shutdown_connections 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 --2- 172.21.15.67:0/1454356257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4104b90 0x7f4fb4107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 --2- 172.21.15.67:0/1454356257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb4071bc0 0x7f4fb4070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 --2- 172.21.15.67:0/1454356257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4070570 0x7f4fb40709c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 -- 172.21.15.67:0/1454356257 >> 172.21.15.67:0/1454356257 conn(0x7f4fb40f8410 msgr2=0x7f4fb40fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 -- 172.21.15.67:0/1454356257 shutdown_connections 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 -- 172.21.15.67:0/1454356257 wait complete. 2023-10-07T23:17:15.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.538+0000 7f4fb999d700 1 Processor -- start 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 -- start start 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 0x7f4fb410fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb4071bc0 0x7f4fb4110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4104b90 0x7f4fb41171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41177b0 con 0x7f4fb4070570 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4117920 con 0x7f4fb4104b90 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb999d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4117ae0 con 0x7f4fb4071bc0 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 0x7f4fb410fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 0x7f4fb410fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38026/0 (socket says 172.21.15.67:38026) 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 -- 172.21.15.67:0/3485829005 learned_addr learned my addr 172.21.15.67:0/3485829005 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb919c700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4104b90 0x7f4fb41171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb3fff700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb4071bc0 0x7f4fb4110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 -- 172.21.15.67:0/3485829005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb4071bc0 msgr2=0x7f4fb4110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb4071bc0 0x7f4fb4110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 -- 172.21.15.67:0/3485829005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4104b90 msgr2=0x7f4fb41171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4104b90 0x7f4fb41171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.539+0000 7f4fb899b700 1 -- 172.21.15.67:0/3485829005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0005040 con 0x7f4fb4070570 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.540+0000 7f4fb899b700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 0x7f4fb410fbd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800ebf0 tx=0x7f4fa800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.540+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8007cc0 con 0x7f4fb4070570 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.540+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb4117d60 con 0x7f4fb4070570 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.540+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fa800f040 con 0x7f4fb4070570 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.540+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8009460 con 0x7f4fb4070570 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.540+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb4118220 con 0x7f4fb4070570 2023-10-07T23:17:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.541+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb41032e0 con 0x7f4fb4070570 2023-10-07T23:17:15.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.543+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f4fa800c680 con 0x7f4fb4070570 2023-10-07T23:17:15.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.544+0000 7f4fb17fa700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4fa409b770 0x7f4fa409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.544+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f4fa80c4e50 con 0x7f4fb4070570 2023-10-07T23:17:15.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.544+0000 7f4fb3fff700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4fa409b770 0x7f4fa409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.545+0000 7f4fb3fff700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4fa409b770 0x7f4fa409dc20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00045a0 tx=0x7f4fa0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:15.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.548+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4fa8090300 con 0x7f4fb4070570 2023-10-07T23:17:15.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.752+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4fb402b190 con 0x7f4fb4070570 2023-10-07T23:17:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.752+0000 7f4fb17fa700 1 -- 172.21.15.67:0/3485829005 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4fa8019020 con 0x7f4fb4070570 2023-10-07T23:17:15.753 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:15.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.754+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4fa409b770 msgr2=0x7f4fa409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.754+0000 7f4fb999d700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4fa409b770 0x7f4fa409dc20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00045a0 tx=0x7f4fa0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:15.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.754+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 msgr2=0x7f4fb410fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.754+0000 7f4fb999d700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 0x7f4fb410fbd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800ebf0 tx=0x7f4fa800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 shutdown_connections 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb4104b90 0x7f4fb41171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4fa409b770 0x7f4fa409dc20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb4070570 0x7f4fb410fbd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 --2- 172.21.15.67:0/3485829005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb4071bc0 0x7f4fb4110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 >> 172.21.15.67:0/3485829005 conn(0x7f4fb40f8410 msgr2=0x7f4fb40fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 shutdown_connections 2023-10-07T23:17:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.755+0000 7f4fb999d700 1 -- 172.21.15.67:0/3485829005 wait complete. 2023-10-07T23:17:15.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:17:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 -- 172.21.15.67:0/3173212312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fccc0 msgr2=0x7f95480fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 --2- 172.21.15.67:0/3173212312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fccc0 0x7f95480fd0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95380049c0 tx=0x7f9538009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 -- 172.21.15.67:0/3173212312 shutdown_connections 2023-10-07T23:17:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 --2- 172.21.15.67:0/3173212312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fccc0 0x7f95480fd0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 --2- 172.21.15.67:0/3173212312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480fe860 0x7f9548105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 --2- 172.21.15.67:0/3173212312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95480fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 -- 172.21.15.67:0/3173212312 >> 172.21.15.67:0/3173212312 conn(0x7f95480f8420 msgr2=0x7f95480fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:15.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 -- 172.21.15.67:0/3173212312 shutdown_connections 2023-10-07T23:17:15.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.962+0000 7f9547fff700 1 -- 172.21.15.67:0/3173212312 wait complete. 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.963+0000 7f9547fff700 1 Processor -- start 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.963+0000 7f9547fff700 1 -- start start 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9547fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480fccc0 0x7f954810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9547fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95481100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9547fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fe860 0x7f9548117180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9547fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548117790 con 0x7f95480fded0 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9547fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548117900 con 0x7f95480fe860 2023-10-07T23:17:15.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9547fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548117ac0 con 0x7f95480fccc0 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95481100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95481100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38042/0 (socket says 172.21.15.67:38042) 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 -- 172.21.15.67:0/3964154068 learned_addr learned my addr 172.21.15.67:0/3964154068 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f9546ffd700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480fccc0 0x7f954810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95477fe700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fe860 0x7f9548117180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 -- 172.21.15.67:0/3964154068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480fccc0 msgr2=0x7f954810fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480fccc0 0x7f954810fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 -- 172.21.15.67:0/3964154068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fe860 msgr2=0x7f9548117180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fe860 0x7f9548117180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95467fc700 1 -- 172.21.15.67:0/3964154068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c005040 con 0x7f95480fded0 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.964+0000 7f95477fe700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fe860 0x7f9548117180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.965+0000 7f95467fc700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95481100f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f953c009750 tx=0x7f953c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.965+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c007c80 con 0x7f95480fded0 2023-10-07T23:17:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.965+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9538005040 con 0x7f95480fded0 2023-10-07T23:17:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.965+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f953c00f040 con 0x7f95480fded0 2023-10-07T23:17:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.965+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c013c60 con 0x7f95480fded0 2023-10-07T23:17:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.965+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481180f0 con 0x7f95480fded0 2023-10-07T23:17:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.966+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954810a540 con 0x7f95480fded0 2023-10-07T23:17:15.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.968+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f953c00e840 con 0x7f95480fded0 2023-10-07T23:17:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.969+0000 7f952f7fe700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f953009b760 0x7f953009dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.969+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f953c0c5860 con 0x7f95480fded0 2023-10-07T23:17:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.969+0000 7f9546ffd700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f953009b760 0x7f953009dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.970+0000 7f9546ffd700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f953009b760 0x7f953009dc10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9538004890 tx=0x7f9538009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:15.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:15.971+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f953c090d10 con 0x7f95480fded0 2023-10-07T23:17:16.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.223+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f954802b0e0 con 0x7f95480fded0 2023-10-07T23:17:16.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.224+0000 7f952f7fe700 1 -- 172.21.15.67:0/3964154068 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f953c094330 con 0x7f95480fded0 2023-10-07T23:17:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:16.225 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:17:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f953009b760 msgr2=0x7f953009dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f953009b760 0x7f953009dc10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9538004890 tx=0x7f9538009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 msgr2=0x7f95481100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95481100f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f953c009750 tx=0x7f953c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 shutdown_connections 2023-10-07T23:17:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480fe860 0x7f9548117180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f953009b760 0x7f953009dc10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95480fccc0 0x7f954810fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 --2- 172.21.15.67:0/3964154068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95480fded0 0x7f95481100f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 >> 172.21.15.67:0/3964154068 conn(0x7f95480f8420 msgr2=0x7f9548103780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 shutdown_connections 2023-10-07T23:17:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.227+0000 7f9547fff700 1 -- 172.21.15.67:0/3964154068 wait complete. 2023-10-07T23:17:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:17:16.243 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:16 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3300360731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:16.243 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:16 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3485829005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:16.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:17:16.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:16 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3300360731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:16.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:16 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3485829005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 -- 172.21.15.67:0/2794066078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fe880 msgr2=0x7eff54105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 --2- 172.21.15.67:0/2794066078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fe880 0x7eff54105140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009d60 tx=0x7eff4c004d10 comp rx=0 tx=0).stop 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 -- 172.21.15.67:0/2794066078 shutdown_connections 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 --2- 172.21.15.67:0/2794066078 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fe880 0x7eff54105140 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 --2- 172.21.15.67:0/2794066078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff540fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 --2- 172.21.15.67:0/2794066078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff540fdef0 0x7eff540fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 -- 172.21.15.67:0/2794066078 >> 172.21.15.67:0/2794066078 conn(0x7eff540f8460 msgr2=0x7eff540fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 -- 172.21.15.67:0/2794066078 shutdown_connections 2023-10-07T23:17:16.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.434+0000 7eff5939d700 1 -- 172.21.15.67:0/2794066078 wait complete. 2023-10-07T23:17:16.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 Processor -- start 2023-10-07T23:17:16.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 -- start start 2023-10-07T23:17:16.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff5410d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fdef0 0x7eff5410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff540fe880 0x7eff54114db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff541153c0 con 0x7eff540fdef0 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff54115530 con 0x7eff540fe880 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff5939d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff541156f0 con 0x7eff540fcce0 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff53fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff5410d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff53fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff5410d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:51298/0 (socket says 172.21.15.67:51298) 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff53fff700 1 -- 172.21.15.67:0/1647813621 learned_addr learned my addr 172.21.15.67:0/1647813621 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff537fe700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fdef0 0x7eff5410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.435+0000 7eff58b9c700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff540fe880 0x7eff54114db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff53fff700 1 -- 172.21.15.67:0/1647813621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff540fe880 msgr2=0x7eff54114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff53fff700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff540fe880 0x7eff54114db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff53fff700 1 -- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fdef0 msgr2=0x7eff5410dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff53fff700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fdef0 0x7eff5410dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff53fff700 1 -- 172.21.15.67:0/1647813621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c009940 con 0x7eff540fcce0 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff537fe700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fdef0 0x7eff5410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff53fff700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff5410d7e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff4400ebc0 tx=0x7eff4400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff440047d0 con 0x7eff540fcce0 2023-10-07T23:17:16.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff54115970 con 0x7eff540fcce0 2023-10-07T23:17:16.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff4400f040 con 0x7eff540fcce0 2023-10-07T23:17:16.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.436+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400c930 con 0x7eff540fcce0 2023-10-07T23:17:16.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.437+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff54115ee0 con 0x7eff540fcce0 2023-10-07T23:17:16.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.437+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff54104020 con 0x7eff540fcce0 2023-10-07T23:17:16.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.440+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7eff44041b00 con 0x7eff540fcce0 2023-10-07T23:17:16.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.441+0000 7eff50ff9700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7eff3c09b770 0x7eff3c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.441+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7eff440c4f50 con 0x7eff540fcce0 2023-10-07T23:17:16.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.441+0000 7eff537fe700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7eff3c09b770 0x7eff3c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.442+0000 7eff537fe700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7eff3c09b770 0x7eff3c09dc20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff48009b80 tx=0x7eff4800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.443+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7eff44090400 con 0x7eff540fcce0 2023-10-07T23:17:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:16 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3300360731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:16.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:16 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3485829005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.645+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff5402b2c0 con 0x7eff540fcce0 2023-10-07T23:17:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.647+0000 7eff50ff9700 1 -- 172.21.15.67:0/1647813621 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7eff44019070 con 0x7eff540fcce0 2023-10-07T23:17:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:16.648 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:17:16.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7eff3c09b770 msgr2=0x7eff3c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7eff3c09b770 0x7eff3c09dc20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff48009b80 tx=0x7eff4800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 msgr2=0x7eff5410d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff5410d7e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff4400ebc0 tx=0x7eff4400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 shutdown_connections 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff540fe880 0x7eff54114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7eff3c09b770 0x7eff3c09dc20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff540fcce0 0x7eff5410d7e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 --2- 172.21.15.67:0/1647813621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff540fdef0 0x7eff5410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 >> 172.21.15.67:0/1647813621 conn(0x7eff540f8460 msgr2=0x7eff541011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 shutdown_connections 2023-10-07T23:17:16.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.650+0000 7eff5939d700 1 -- 172.21.15.67:0/1647813621 wait complete. 2023-10-07T23:17:16.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-07T23:15:26.869397+0000', 'modified': '2023-10-07T23:15:55.854427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24538, 'mds_2': 24521, 'mds_3': 24467, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14502': {'gid': 14502, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/3010063667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 3010063667}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 3010063667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/1133310774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 1133310774}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 1133310774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.123:6833/1348503065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1348503065}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1348503065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.67:6837/909177824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6836', 'nonce': 909177824}, {'type': 'v1', 'addr': '172.21.15.67:6837', 'nonce': 909177824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/3322109112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 3322109112}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 3322109112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/4209275840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 4209275840}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 4209275840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/1647886755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1647886755}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1647886755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.86:6837/4272295866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 4272295866}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 4272295866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6839/3252673889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 3252673889}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 3252673889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6839/2147538694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 2147538694}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 2147538694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T23:17:16.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-07T23:17:16.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-10-07T23:17:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 -- 172.21.15.67:0/1357887779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fcce0 msgr2=0x7f44b00fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 --2- 172.21.15.67:0/1357887779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fcce0 0x7f44b00fd0f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0049c0 tx=0x7f44ac009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 -- 172.21.15.67:0/1357887779 shutdown_connections 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 --2- 172.21.15.67:0/1357887779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b00fe880 0x7f44b0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 --2- 172.21.15.67:0/1357887779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fcce0 0x7f44b00fd0f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 --2- 172.21.15.67:0/1357887779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fdef0 0x7f44b00fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 -- 172.21.15.67:0/1357887779 >> 172.21.15.67:0/1357887779 conn(0x7f44b00f8460 msgr2=0x7f44b00fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 -- 172.21.15.67:0/1357887779 shutdown_connections 2023-10-07T23:17:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.860+0000 7f44b639d700 1 -- 172.21.15.67:0/1357887779 wait complete. 2023-10-07T23:17:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 Processor -- start 2023-10-07T23:17:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 -- start start 2023-10-07T23:17:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 0x7f44b010fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fdef0 0x7f44b0110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b00fe880 0x7f44b0117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b01177d0 con 0x7f44b00fdef0 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0117940 con 0x7f44b00fcce0 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b639d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0117b00 con 0x7f44b00fe880 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b4b9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fdef0 0x7f44b0110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b539b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 0x7f44b010fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b539b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 0x7f44b010fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39696/0 (socket says 172.21.15.67:39696) 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b539b700 1 -- 172.21.15.67:0/2003868287 learned_addr learned my addr 172.21.15.67:0/2003868287 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.861+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b00fe880 0x7f44b0117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b4b9a700 1 -- 172.21.15.67:0/2003868287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b00fe880 msgr2=0x7f44b0117190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b00fe880 0x7f44b0117190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b4b9a700 1 -- 172.21.15.67:0/2003868287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 msgr2=0x7f44b010fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 0x7f44b010fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b4b9a700 1 -- 172.21.15.67:0/2003868287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0005040 con 0x7f44b00fdef0 2023-10-07T23:17:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b539b700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 0x7f44b010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:16.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fdef0 0x7f44b0110140 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009520 tx=0x7f44a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44a5ffb700 1 -- 172.21.15.67:0/2003868287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0007be0 con 0x7f44b00fdef0 2023-10-07T23:17:16.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac005040 con 0x7f44b00fdef0 2023-10-07T23:17:16.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44a5ffb700 1 -- 172.21.15.67:0/2003868287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a000f040 con 0x7f44b00fdef0 2023-10-07T23:17:16.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44a5ffb700 1 -- 172.21.15.67:0/2003868287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0013bd0 con 0x7f44b00fdef0 2023-10-07T23:17:16.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.862+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b0118130 con 0x7f44b00fdef0 2023-10-07T23:17:16.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.865+0000 7f44a5ffb700 1 -- 172.21.15.67:0/2003868287 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f44a000e7e0 con 0x7f44b00fdef0 2023-10-07T23:17:16.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 Processor -- start 2023-10-07T23:17:16.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 -- start start 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387440 0x561c71387850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 0x561c713886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44a5ffb700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f449c09b770 0x7f449c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c71389140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c712dca70 con 0x561c71388cf0 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c712da270 con 0x561c71387440 2023-10-07T23:17:16.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c712d6060 con 0x561c71387d90 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44a5ffb700 1 -- 172.21.15.67:0/2003868287 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f44a00c4ff0 con 0x7f44b00fdef0 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44b539b700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f449c09b770 0x7f449c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44b5b9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c71389140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.867+0000 7f44b4b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 0x561c713886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 0x561c713886d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:51328/0 (socket says 172.21.15.67:51328) 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3287585772 learned_addr learned my addr 172.21.15.67:0/3287585772 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387440 0x561c71387850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3287585772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387440 msgr2=0x561c71387850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387440 0x561c71387850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3287585772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 msgr2=0x561c71389140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c71389140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3287585772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c71170270 con 0x561c71387d90 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b539b700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f449c09b770 0x7f449c09dc20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f44ac013a40 tx=0x7f44ac009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 0x561c713886d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44a0090bd0 tx=0x7f44a000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d112cc950e2c046 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f449affd700 1 -- 172.21.15.67:0/3287585772 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0094860 con 0x561c71387d90 2023-10-07T23:17:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f449affd700 1 -- 172.21.15.67:0/3287585772 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a0094160 con 0x561c71387d90 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.868+0000 7f449affd700 1 -- 172.21.15.67:0/3287585772 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0092500 con 0x561c71387d90 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3287585772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 msgr2=0x561c713886d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 0x561c713886d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44a0090bd0 tx=0x7f44a000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3287585772 shutdown_connections 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387440 0x561c71387850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c71389140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3287585772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387d90 0x561c713886d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3287585772 >> 172.21.15.67:0/3287585772 conn(0x561c71382470 msgr2=0x561c71382860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3287585772 shutdown_connections 2023-10-07T23:17:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.869+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3287585772 wait complete. 2023-10-07T23:17:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 -- start start 2023-10-07T23:17:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 Processor -- start 2023-10-07T23:17:16.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 0x561c71390c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c713911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71388cf0 0x561c71398230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c71281890 con 0x561c71387440 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7129cea0 con 0x561c71387d90 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c71158e30 con 0x561c71388cf0 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44c5c93b80 10 client.0 _ll_register_callbacks cb 0x561c713492c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.870+0000 7f44b539b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 0x561c71390c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71388cf0 0x561c71398230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b539b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 0x561c71390c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38102/0 (socket says 172.21.15.67:38102) 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b539b700 1 -- 172.21.15.67:0/491883459 learned_addr learned my addr 172.21.15.67:0/491883459 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c713911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 -- 172.21.15.67:0/491883459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 msgr2=0x561c713911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c713911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 -- 172.21.15.67:0/491883459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 msgr2=0x561c71390c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 0x561c71390c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 -- 172.21.15.67:0/491883459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b0118130 con 0x561c71388cf0 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b539b700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 0x561c71390c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71388cf0 0x561c71398230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44a80090e0 tx=0x7f44a8009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0638e972ef52ca server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f448ffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.871+0000 7f448ffff700 1 -- 172.21.15.67:0/491883459 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a800e030 con 0x561c71388cf0 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c7137ace0 con 0x561c71388cf0 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f448ffff700 1 -- 172.21.15.67:0/491883459 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f44a8016930 con 0x561c71388cf0 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f448ffff700 1 -- 172.21.15.67:0/491883459 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a8016d00 con 0x561c71388cf0 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f44c5c93b80 10 client.25234 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561c71112fe0 con 0x561c71388cf0 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f448ffff700 1 -- 172.21.15.67:0/491883459 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c7137ace0 con 0x561c71388cf0 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f448ffff700 1 -- 172.21.15.67:0/491883459 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f44a80193d0 con 0x561c71388cf0 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.872+0000 7f44c5c93b80 10 client.25234 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44c5c93b80 10 client.25234 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44c5c93b80 20 client.25234 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c7139bef0 0x561c713a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44c5c93b80 4 client.25234 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:17:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 --> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c71357ab0 con 0x561c7139bef0 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44b539b700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c7139bef0 0x561c713a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f448ffff700 10 client.25234 ms_handle_connect on v2:172.21.15.123:6832/1348503065 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.873+0000 7f44b539b700 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c7139bef0 0x561c713a11d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c38e9ed0829eee server_cookie=3021ef2b473319bf in_seq=0 out_seq=0 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.874+0000 7f448ffff700 1 -- 172.21.15.67:0/491883459 <== mds.4 v2:172.21.15.123:6832/1348503065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561c71357ab0 con 0x561c7139bef0 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.874+0000 7f448ffff700 10 client.25234 handle_command_reply: tid=0 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.874+0000 7f44c5c93b80 1 client.25234 shutdown 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71388cf0 msgr2=0x561c71398230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71388cf0 0x561c71398230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44a80090e0 tx=0x7f44a8009040 comp rx=0 tx=0).stop 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 shutdown_connections 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c7139bef0 0x561c713a11d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71388cf0 0x561c71398230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f448ffff700 0 client.25234 ms_handle_reset on v2:172.21.15.123:6832/1348503065 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387440 0x561c71390c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/491883459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c713911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 >> 172.21.15.67:0/491883459 conn(0x561c71382470 msgr2=0x561c7137cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 shutdown_connections 2023-10-07T23:17:16.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 1 -- 172.21.15.67:0/491883459 wait complete. 2023-10-07T23:17:16.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.875+0000 7f44c5c93b80 20 client.25234 trim_cache size 0 max 16384 2023-10-07T23:17:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.892+0000 7f44c5c93b80 1 Processor -- start 2023-10-07T23:17:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.892+0000 7f44c5c93b80 1 -- start start 2023-10-07T23:17:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c7138b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c7138be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c7138eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c71357ab0 con 0x561c71388cf0 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c712a1d60 con 0x561c71387d90 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c712822b0 con 0x561c71387440 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c7138eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c7138eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38110/0 (socket says 172.21.15.67:38110) 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 -- 172.21.15.67:0/2070279882 learned_addr learned my addr 172.21.15.67:0/2070279882 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b539b700 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c7138b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c7138be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 -- 172.21.15.67:0/2070279882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 msgr2=0x561c7138b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c7138b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 -- 172.21.15.67:0/2070279882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 msgr2=0x561c7138be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c7138be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.893+0000 7f44b5b9c700 1 -- 172.21.15.67:0/2070279882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c712a4780 con 0x561c71388cf0 2023-10-07T23:17:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c7138eb90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f44a8004b00 tx=0x7f44a8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb46a7f94aa1d53 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f449b7fe700 1 -- 172.21.15.67:0/2070279882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a800e050 con 0x561c71388cf0 2023-10-07T23:17:16.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f449b7fe700 1 -- 172.21.15.67:0/2070279882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a800f040 con 0x561c71388cf0 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f449b7fe700 1 -- 172.21.15.67:0/2070279882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a801a470 con 0x561c71388cf0 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 -- 172.21.15.67:0/2070279882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 msgr2=0x561c7138eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c7138eb90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f44a8004b00 tx=0x7f44a8009040 comp rx=0 tx=0).stop 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 -- 172.21.15.67:0/2070279882 shutdown_connections 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71388cf0 0x561c7138eb90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c7138b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/2070279882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71387d90 0x561c7138be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.894+0000 7f44c5c93b80 1 -- 172.21.15.67:0/2070279882 >> 172.21.15.67:0/2070279882 conn(0x561c71382470 msgr2=0x561c71382840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.895+0000 7f44c5c93b80 1 -- 172.21.15.67:0/2070279882 shutdown_connections 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.895+0000 7f44c5c93b80 1 -- 172.21.15.67:0/2070279882 wait complete. 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.895+0000 7f44c5c93b80 1 -- start start 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.895+0000 7f44c5c93b80 1 Processor -- start 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.895+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c713a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.895+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 0x561c713a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44c5c93b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71388cf0 0x561c713eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c713ee330 con 0x561c71387d90 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c713ee470 con 0x561c71388cf0 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44c5c93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c713ee5e0 con 0x561c71387440 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44c5c93b80 10 client.0 _ll_register_callbacks cb 0x561c713492c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 0x561c713a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 0x561c713a2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38114/0 (socket says 172.21.15.67:38114) 2023-10-07T23:17:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3782351630 learned_addr learned my addr 172.21.15.67:0/3782351630 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b539b700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c713a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b5b9c700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71388cf0 0x561c713eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 msgr2=0x561c713a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c713a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3782351630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71388cf0 msgr2=0x561c713eddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71388cf0 0x561c713eddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 -- 172.21.15.67:0/3782351630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c71112fe0 con 0x561c71387d90 2023-10-07T23:17:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f44b4b9a700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 0x561c713a2400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44a0090660 tx=0x7f44a007f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8c8fd5ae3c74cc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.896+0000 7f448ffff700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f448ffff700 1 -- 172.21.15.67:0/3782351630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a00904a0 con 0x561c71387d90 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44a0094160 con 0x561c71387d90 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f448ffff700 1 -- 172.21.15.67:0/3782351630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f44a0094860 con 0x561c71387d90 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f448ffff700 1 -- 172.21.15.67:0/3782351630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a00973b0 con 0x561c71387d90 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 10 client.15189 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561c7128b470 con 0x561c71387d90 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f448ffff700 1 -- 172.21.15.67:0/3782351630 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a0094160 con 0x561c71387d90 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f448ffff700 1 -- 172.21.15.67:0/3782351630 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f44a0093650 con 0x561c71387d90 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 10 client.15189 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 20 client.15189 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c713f3030 0x561c713ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 4 client.15189 mds_command: new command op to 24448 tid=0[{"prefix": "status"}] 2023-10-07T23:17:16.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.897+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 --> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561c71278e00 con 0x561c713f3030 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44b539b700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c713f3030 0x561c713ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44b539b700 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c713f3030 0x561c713ef960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f90a88ae611a446 server_cookie=265d7d266e118b48 in_seq=0 out_seq=0 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f448ffff700 10 client.15189 ms_handle_connect on v2:172.21.15.123:6832/1348503065 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f448ffff700 1 -- 172.21.15.67:0/3782351630 <== mds.4 v2:172.21.15.123:6832/1348503065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561c71278e00 con 0x561c713f3030 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f448ffff700 10 client.15189 handle_command_reply: tid=0 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 client.15189 shutdown 2023-10-07T23:17:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 msgr2=0x561c713a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 0x561c713a2400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44a0090660 tx=0x7f44a007f040 comp rx=0 tx=0).stop 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 shutdown_connections 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:6832/1348503065,v1:172.21.15.123:6833/1348503065] conn(0x561c713f3030 0x561c713ef960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c71388cf0 0x561c713eddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c71387440 0x561c713a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 --2- 172.21.15.67:0/3782351630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c71387d90 0x561c713a2400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f448ffff700 0 client.15189 ms_handle_reset on v2:172.21.15.123:6832/1348503065 2023-10-07T23:17:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.898+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 >> 172.21.15.67:0/3782351630 conn(0x561c71382470 msgr2=0x561c7138b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 shutdown_connections 2023-10-07T23:17:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44c5c93b80 1 -- 172.21.15.67:0/3782351630 wait complete. 2023-10-07T23:17:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44c5c93b80 20 client.15189 trim_cache size 0 max 16384 2023-10-07T23:17:16.909 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:17:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:17:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24448, 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 82.048414445000006, 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:17:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:17:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 147.115426649 2023-10-07T23:17:16.911 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:17:16.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f449c09b770 msgr2=0x7f449c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f449c09b770 0x7f449c09dc20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f44ac013a40 tx=0x7f44ac009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fdef0 msgr2=0x7f44b0110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fdef0 0x7f44b0110140 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009520 tx=0x7f44a0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 shutdown_connections 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b00fcce0 0x7f44b010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f449c09b770 0x7f449c09dc20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b00fe880 0x7f44b0117190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 --2- 172.21.15.67:0/2003868287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b00fdef0 0x7f44b0110140 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.899+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 >> 172.21.15.67:0/2003868287 conn(0x7f44b00f8460 msgr2=0x7f44b00fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.900+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 shutdown_connections 2023-10-07T23:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:16.900+0000 7f44b639d700 1 -- 172.21.15.67:0/2003868287 wait complete. 2023-10-07T23:17:16.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 -- 172.21.15.67:0/274752852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc071bc0 msgr2=0x7efcdc070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 --2- 172.21.15.67:0/274752852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc071bc0 0x7efcdc070030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efccc0049c0 tx=0x7efccc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 -- 172.21.15.67:0/274752852 shutdown_connections 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 --2- 172.21.15.67:0/274752852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcdc104b80 0x7efcdc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 --2- 172.21.15.67:0/274752852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc071bc0 0x7efcdc070030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 --2- 172.21.15.67:0/274752852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcdc070570 0x7efcdc0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 -- 172.21.15.67:0/274752852 >> 172.21.15.67:0/274752852 conn(0x7efcdc0f8460 msgr2=0x7efcdc0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 -- 172.21.15.67:0/274752852 shutdown_connections 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.109+0000 7efcdb59e700 1 -- 172.21.15.67:0/274752852 wait complete. 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 Processor -- start 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 -- start start 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 0x7efcdc10fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcdc071bc0 0x7efcdc1100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcdc104b80 0x7efcdc117140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc117750 con 0x7efcdc070570 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1178c0 con 0x7efcdc071bc0 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcdb59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc117a80 con 0x7efcdc104b80 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcda59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 0x7efcdc10fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcda59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 0x7efcdc10fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38136/0 (socket says 172.21.15.67:38136) 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcda59c700 1 -- 172.21.15.67:0/1510563750 learned_addr learned my addr 172.21.15.67:0/1510563750 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.110+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcdc071bc0 0x7efcdc1100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcdad9d700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcdc104b80 0x7efcdc117140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcda59c700 1 -- 172.21.15.67:0/1510563750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcdc104b80 msgr2=0x7efcdc117140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcda59c700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcdc104b80 0x7efcdc117140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcda59c700 1 -- 172.21.15.67:0/1510563750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcdc071bc0 msgr2=0x7efcdc1100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcda59c700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcdc071bc0 0x7efcdc1100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcda59c700 1 -- 172.21.15.67:0/1510563750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0005040 con 0x7efcdc070570 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcda59c700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 0x7efcdc10fb70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b3b0 tx=0x7efccc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efccaffd700 1 -- 172.21.15.67:0/1510563750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00e040 con 0x7efcdc070570 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efccaffd700 1 -- 172.21.15.67:0/1510563750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efccc013420 con 0x7efcdc070570 2023-10-07T23:17:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efccc005040 con 0x7efcdc070570 2023-10-07T23:17:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efccaffd700 1 -- 172.21.15.67:0/1510563750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc01c660 con 0x7efcdc070570 2023-10-07T23:17:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.111+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc118020 con 0x7efcdc070570 2023-10-07T23:17:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.115+0000 7efccaffd700 1 -- 172.21.15.67:0/1510563750 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7efccc013590 con 0x7efcdc070570 2023-10-07T23:17:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 Processor -- start 2023-10-07T23:17:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 -- start start 2023-10-07T23:17:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efccaffd700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efcc409b7f0 0x7efcc409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acb1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635acb26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb2cf0 0x55635acb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635ac06a70 con 0x55635acb1440 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635ac04270 con 0x55635acb2cf0 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efcef097b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635ac00060 con 0x55635acb1d90 2023-10-07T23:17:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.116+0000 7efccaffd700 1 -- 172.21.15.67:0/1510563750 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7efccc0c63a0 con 0x7efcdc070570 2023-10-07T23:17:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acb1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635acb26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acb1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38138/0 (socket says 172.21.15.67:38138) 2023-10-07T23:17:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 -- 172.21.15.67:0/4148265982 learned_addr learned my addr 172.21.15.67:0/4148265982 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcdad9d700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efcc409b7f0 0x7efcc409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcdad9d700 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb2cf0 0x55635acb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 -- 172.21.15.67:0/4148265982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 msgr2=0x55635acb26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635acb26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 -- 172.21.15.67:0/4148265982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb2cf0 msgr2=0x55635acb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb2cf0 0x55635acb3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 -- 172.21.15.67:0/4148265982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55635aa9a270 con 0x55635acb1440 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635acb26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcdad9d700 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb2cf0 0x55635acb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.117+0000 7efcda59c700 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acb1850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efccc091a10 tx=0x7efccc0c3a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93229d793ed2021 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcdad9d700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efcc409b7f0 0x7efcc409dca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009fd0 tx=0x7efcd4009df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcc3fff700 1 -- 172.21.15.67:0/4148265982 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc094e70 con 0x55635acb1440 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcc3fff700 1 -- 172.21.15.67:0/4148265982 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efccc0a67d0 con 0x55635acb1440 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcc3fff700 1 -- 172.21.15.67:0/4148265982 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc0a6270 con 0x55635acb1440 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 -- 172.21.15.67:0/4148265982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 msgr2=0x55635acb1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acb1850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efccc091a10 tx=0x7efccc0c3a00 comp rx=0 tx=0).stop 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 -- 172.21.15.67:0/4148265982 shutdown_connections 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb2cf0 0x55635acb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acb1850 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 --2- 172.21.15.67:0/4148265982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635acb26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.118+0000 7efcef097b80 1 -- 172.21.15.67:0/4148265982 >> 172.21.15.67:0/4148265982 conn(0x55635acac470 msgr2=0x55635acac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.119+0000 7efcef097b80 1 -- 172.21.15.67:0/4148265982 shutdown_connections 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.119+0000 7efcef097b80 1 -- 172.21.15.67:0/4148265982 wait complete. 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.119+0000 7efcef097b80 1 -- start start 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.119+0000 7efcef097b80 1 Processor -- start 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acbaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1d90 0x55635acbaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acc2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635abab890 con 0x55635acb1440 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635abc6ea0 con 0x55635acb1d90 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635aa82e30 con 0x55635acb2cf0 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcef097b80 10 client.0 _ll_register_callbacks cb 0x55635ac732c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acbaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1d90 0x55635acbaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcdad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acc2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acbaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38152/0 (socket says 172.21.15.67:38152) 2023-10-07T23:17:17.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 -- 172.21.15.67:0/3940553747 learned_addr learned my addr 172.21.15.67:0/3940553747 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1d90 0x55635acbaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39738/0 (socket says 172.21.15.67:39738) 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 -- 172.21.15.67:0/3940553747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 msgr2=0x55635acc2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acc2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 -- 172.21.15.67:0/3940553747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1d90 msgr2=0x55635acbaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1d90 0x55635acbaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.120+0000 7efcda59c700 1 -- 172.21.15.67:0/3940553747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd00047e0 con 0x55635acb1440 2023-10-07T23:17:17.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcda59c700 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acbaab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efccc095210 tx=0x7efccc0c3a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23e485565076d3d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcc17fa700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcc17fa700 1 -- 172.21.15.67:0/3940553747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc094e70 con 0x55635acb1440 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcc17fa700 1 -- 172.21.15.67:0/3940553747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7efccc095510 con 0x55635acb1440 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55635aca4ce0 con 0x55635acb1440 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcc17fa700 1 -- 172.21.15.67:0/3940553747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efccc098760 con 0x55635acb1440 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcef097b80 10 client.15201 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7efcdc118020 con 0x55635acb1440 2023-10-07T23:17:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.121+0000 7efcc17fa700 1 -- 172.21.15.67:0/3940553747 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55635aca4ce0 con 0x55635acb1440 2023-10-07T23:17:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcc17fa700 1 -- 172.21.15.67:0/3940553747 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7efccc095770 con 0x55635acb1440 2023-10-07T23:17:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcef097b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcef097b80 10 client.15201 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-07T23:17:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcef097b80 20 client.15201 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcef097b80 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635acc5700 0x55635acca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcef097b80 4 client.15201 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 --> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55635ac81ab0 con 0x55635acc5700 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635acc5700 0x55635acca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635acc5700 0x55635acca9a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9f70b1fec1e0751 server_cookie=26252e510d7380d7 in_seq=0 out_seq=0 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.122+0000 7efcc17fa700 10 client.15201 ms_handle_connect on v2:172.21.15.67:6836/909177824 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcc17fa700 1 -- 172.21.15.67:0/3940553747 <== mds.3 v2:172.21.15.67:6836/909177824 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55635ac81ab0 con 0x55635acc5700 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcc17fa700 10 client.15201 handle_command_reply: tid=0 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 client.15201 shutdown 2023-10-07T23:17:17.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 msgr2=0x55635acbaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acbaab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efccc095210 tx=0x7efccc0c3a00 comp rx=0 tx=0).stop 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 shutdown_connections 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635acc5700 0x55635acca9a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acc2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1440 0x55635acbaab0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcc17fa700 0 client.15201 ms_handle_reset on v2:172.21.15.67:6836/909177824 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 --2- 172.21.15.67:0/3940553747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1d90 0x55635acbaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.123+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 >> 172.21.15.67:0/3940553747 conn(0x55635acac470 msgr2=0x55635acadef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.124+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 shutdown_connections 2023-10-07T23:17:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.124+0000 7efcef097b80 1 -- 172.21.15.67:0/3940553747 wait complete. 2023-10-07T23:17:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.124+0000 7efcef097b80 20 client.15201 trim_cache size 0 max 16384 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 Processor -- start 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 -- start start 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635acb92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 0x55635acad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635ac06a70 con 0x55635acb1d90 2023-10-07T23:17:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635acc6f50 con 0x55635acb1440 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.141+0000 7efcef097b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635acbb9d0 con 0x55635acb2cf0 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 0x55635acad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 0x55635acad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38168/0 (socket says 172.21.15.67:38168) 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 -- 172.21.15.67:0/4153214791 learned_addr learned my addr 172.21.15.67:0/4153214791 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcda59c700 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635acb92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcdad9d700 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 -- 172.21.15.67:0/4153214791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 msgr2=0x55635acad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 -- 172.21.15.67:0/4153214791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 msgr2=0x55635acb92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635acb92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 -- 172.21.15.67:0/4153214791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55635aa3cfe0 con 0x55635acb1d90 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcda59c700 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635acb92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 0x55635acad4b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55635acbbb30 tx=0x7efcd0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e715ed7304dc3b6a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcc8ff9700 1 -- 172.21.15.67:0/4153214791 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd001d070 con 0x55635acb1d90 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcc8ff9700 1 -- 172.21.15.67:0/4153214791 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd000f040 con 0x55635acb1d90 2023-10-07T23:17:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.142+0000 7efcc8ff9700 1 -- 172.21.15.67:0/4153214791 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000e040 con 0x55635acb1d90 2023-10-07T23:17:17.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 -- 172.21.15.67:0/4153214791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 msgr2=0x55635acad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 0x55635acad4b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55635acbbb30 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 -- 172.21.15.67:0/4153214791 shutdown_connections 2023-10-07T23:17:17.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635acb92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb2cf0 0x55635acad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 --2- 172.21.15.67:0/4153214791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb1d90 0x55635acad4b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 -- 172.21.15.67:0/4153214791 >> 172.21.15.67:0/4153214791 conn(0x55635acac470 msgr2=0x55635acb8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 -- 172.21.15.67:0/4153214791 shutdown_connections 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 -- 172.21.15.67:0/4153214791 wait complete. 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.143+0000 7efcef097b80 1 -- start start 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 Processor -- start 2023-10-07T23:17:17.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635ad135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635ad15af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb2cf0 0x55635ad18040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635acb8410 con 0x55635acb2cf0 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635ad18580 con 0x55635acb1440 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635ad186c0 con 0x55635acb1d90 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcef097b80 10 client.0 _ll_register_callbacks cb 0x55635ac732c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcda59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635ad135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcdad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb2cf0 0x55635ad18040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcda59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635ad135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39762/0 (socket says 172.21.15.67:39762) 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcda59c700 1 -- 172.21.15.67:0/2167874716 learned_addr learned my addr 172.21.15.67:0/2167874716 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.144+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635ad15af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcda59c700 1 -- 172.21.15.67:0/2167874716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 msgr2=0x55635ad15af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcda59c700 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635ad15af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcda59c700 1 -- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb2cf0 msgr2=0x55635ad18040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcda59c700 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb2cf0 0x55635ad18040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcda59c700 1 -- 172.21.15.67:0/2167874716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd00047e0 con 0x55635acb1440 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcda59c700 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635ad135a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efccc004890 tx=0x7efccc0c3a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e5a96ae218199 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcc17fa700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcc17fa700 1 -- 172.21.15.67:0/2167874716 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc095980 con 0x55635acb1440 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcc17fa700 1 -- 172.21.15.67:0/2167874716 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7efccc0ac050 con 0x55635acb1440 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efccc0a67d0 con 0x55635acb1440 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcc17fa700 1 -- 172.21.15.67:0/2167874716 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efccc094e70 con 0x55635acb1440 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcef097b80 10 client.25256 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.145+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7efcd00090b0 con 0x55635acb1440 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcc17fa700 1 -- 172.21.15.67:0/2167874716 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efccc0a67d0 con 0x55635acb1440 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcc17fa700 1 -- 172.21.15.67:0/2167874716 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7efccc094a00 con 0x55635acb1440 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcef097b80 10 client.25256 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcef097b80 10 client.25256 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcef097b80 20 client.25256 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcef097b80 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635ad1ce40 0x55635ad19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcef097b80 4 client.25256 mds_command: new command op to 24467 tid=0[{"prefix": "status"}] 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 --> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55635aba2e00 con 0x55635ad1ce40 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.146+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635ad1ce40 0x55635ad19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcc17fa700 10 client.25256 ms_handle_connect on v2:172.21.15.67:6836/909177824 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcd9d9b700 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635ad1ce40 0x55635ad19820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91c594dd3e4cc199 server_cookie=ca13fae4232c91ad in_seq=0 out_seq=0 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcc17fa700 1 -- 172.21.15.67:0/2167874716 <== mds.3 v2:172.21.15.67:6836/909177824 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55635aba2e00 con 0x55635ad1ce40 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcc17fa700 10 client.25256 handle_command_reply: tid=0 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 client.25256 shutdown 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 msgr2=0x55635ad135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635ad135a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efccc004890 tx=0x7efccc0c3a00 comp rx=0 tx=0).stop 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 shutdown_connections 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55635acb1440 0x55635ad135a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:6836/909177824,v1:172.21.15.67:6837/909177824] conn(0x55635ad1ce40 0x55635ad19820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55635acb2cf0 0x55635ad18040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 --2- 172.21.15.67:0/2167874716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55635acb1d90 0x55635ad15af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcc17fa700 0 client.25256 ms_handle_reset on v2:172.21.15.67:6836/909177824 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 >> 172.21.15.67:0/2167874716 conn(0x55635acac470 msgr2=0x55635aca6670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 shutdown_connections 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 1 -- 172.21.15.67:0/2167874716 wait complete. 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.147+0000 7efcef097b80 20 client.25256 trim_cache size 0 max 16384 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24467, 2023-10-07T23:17:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 83.306929284999995, 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:17:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 138.53264074399999 2023-10-07T23:17:17.159 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:17:17.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efcc409b7f0 msgr2=0x7efcc409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efcc409b7f0 0x7efcc409dca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009fd0 tx=0x7efcd4009df0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 msgr2=0x7efcdc10fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 0x7efcdc10fb70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b3b0 tx=0x7efccc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 shutdown_connections 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efcc409b7f0 0x7efcc409dca0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcdc104b80 0x7efcdc117140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcdc070570 0x7efcdc10fb70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 --2- 172.21.15.67:0/1510563750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcdc071bc0 0x7efcdc1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 >> 172.21.15.67:0/1510563750 conn(0x7efcdc0f8460 msgr2=0x7efcdc0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 shutdown_connections 2023-10-07T23:17:17.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.148+0000 7efcdb59e700 1 -- 172.21.15.67:0/1510563750 wait complete. 2023-10-07T23:17:17.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-07T23:17:17.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:17 smithi067 ceph-mon[82723]: pgmap v345: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:17.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:17 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3964154068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:17.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:17 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1647813621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.355+0000 7f6337fff700 1 -- 172.21.15.67:0/1936103229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 msgr2=0x7f63380709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.355+0000 7f6337fff700 1 --2- 172.21.15.67:0/1936103229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f63380709c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f632c0049c0 tx=0x7f632c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 -- 172.21.15.67:0/1936103229 shutdown_connections 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 --2- 172.21.15.67:0/1936103229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338104b80 0x7f6338107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 --2- 172.21.15.67:0/1936103229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6338071bc0 0x7f6338070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 --2- 172.21.15.67:0/1936103229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f63380709c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 -- 172.21.15.67:0/1936103229 >> 172.21.15.67:0/1936103229 conn(0x7f63380f8460 msgr2=0x7f63380fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 -- 172.21.15.67:0/1936103229 shutdown_connections 2023-10-07T23:17:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 -- 172.21.15.67:0/1936103229 wait complete. 2023-10-07T23:17:17.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.356+0000 7f6337fff700 1 Processor -- start 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 -- start start 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f6338100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338071bc0 0x7f6338100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6338104b80 0x7f63380fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380fd330 con 0x7f6338071bc0 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380fd4a0 con 0x7f6338070570 2023-10-07T23:17:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6337fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380fd660 con 0x7f6338104b80 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f6338100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f63367fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338071bc0 0x7f6338100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f6338100020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39778/0 (socket says 172.21.15.67:39778) 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f6336ffd700 1 -- 172.21.15.67:0/1374772991 learned_addr learned my addr 172.21.15.67:0/1374772991 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.357+0000 7f63377fe700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6338104b80 0x7f63380fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6336ffd700 1 -- 172.21.15.67:0/1374772991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6338104b80 msgr2=0x7f63380fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6336ffd700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6338104b80 0x7f63380fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6336ffd700 1 -- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338071bc0 msgr2=0x7f6338100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6336ffd700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338071bc0 0x7f6338100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6336ffd700 1 -- 172.21.15.67:0/1374772991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c005040 con 0x7f6338070570 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6336ffd700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f6338100020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f632800ebf0 tx=0x7f632800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f631f7fe700 1 -- 172.21.15.67:0/1374772991 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328007cc0 con 0x7f6338070570 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f631f7fe700 1 -- 172.21.15.67:0/1374772991 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f632800f040 con 0x7f6338070570 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633811c130 con 0x7f6338070570 2023-10-07T23:17:17.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f631f7fe700 1 -- 172.21.15.67:0/1374772991 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328009460 con 0x7f6338070570 2023-10-07T23:17:17.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.358+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633811c6a0 con 0x7f6338070570 2023-10-07T23:17:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.362+0000 7f631f7fe700 1 -- 172.21.15.67:0/1374772991 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f632800c680 con 0x7f6338070570 2023-10-07T23:17:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.362+0000 7f634bddab80 1 Processor -- start 2023-10-07T23:17:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 -- start start 2023-10-07T23:17:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f4377d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f4377e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f4377f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f436d2a70 con 0x557f4377ecf0 2023-10-07T23:17:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f436d0270 con 0x557f4377dd90 2023-10-07T23:17:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f634bddab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f436cc060 con 0x557f4377d440 2023-10-07T23:17:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f631f7fe700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f632009b7f0 0x7f632009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f63367fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f4377f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f4377e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f63367fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f4377f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38200/0 (socket says 172.21.15.67:38200) 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f63367fc700 1 -- 172.21.15.67:0/569676201 learned_addr learned my addr 172.21.15.67:0/569676201 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f631f7fe700 1 -- 172.21.15.67:0/1374772991 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f63280c4e90 con 0x7f6338070570 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f63377fe700 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f4377d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.363+0000 7f63377fe700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f632009b7f0 0x7f632009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f6336ffd700 1 -- 172.21.15.67:0/569676201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 msgr2=0x557f4377d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f6336ffd700 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f4377d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f6336ffd700 1 -- 172.21.15.67:0/569676201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 msgr2=0x557f4377f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f6336ffd700 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f4377f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f6336ffd700 1 -- 172.21.15.67:0/569676201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f43566270 con 0x557f4377dd90 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f6336ffd700 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f4377e6d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6328090500 tx=0x7f63280c24f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d690748ff173864 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f63377fe700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f632009b7f0 0x7f632009dca0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63300049c0 tx=0x7f633000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f631cff9700 1 -- 172.21.15.67:0/569676201 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328093b40 con 0x557f4377dd90 2023-10-07T23:17:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f631cff9700 1 -- 172.21.15.67:0/569676201 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6328093960 con 0x557f4377dd90 2023-10-07T23:17:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.364+0000 7f631cff9700 1 -- 172.21.15.67:0/569676201 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328094090 con 0x557f4377dd90 2023-10-07T23:17:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 -- 172.21.15.67:0/569676201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 msgr2=0x557f4377e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f4377e6d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6328090500 tx=0x7f63280c24f0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 -- 172.21.15.67:0/569676201 shutdown_connections 2023-10-07T23:17:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f4377f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f4377d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 --2- 172.21.15.67:0/569676201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f4377e6d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 -- 172.21.15.67:0/569676201 >> 172.21.15.67:0/569676201 conn(0x557f43778470 msgr2=0x557f43778860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 -- 172.21.15.67:0/569676201 shutdown_connections 2023-10-07T23:17:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 -- 172.21.15.67:0/569676201 wait complete. 2023-10-07T23:17:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.365+0000 7f634bddab80 1 -- start start 2023-10-07T23:17:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 Processor -- start 2023-10-07T23:17:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377d440 0x557f43786d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 0x557f437872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377ecf0 0x557f4378e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43677890 con 0x557f4377dd90 2023-10-07T23:17:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43692ea0 con 0x557f4377d440 2023-10-07T23:17:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4354ee30 con 0x557f4377ecf0 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f634bddab80 10 client.0 _ll_register_callbacks cb 0x557f4373f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.366+0000 7f63367fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 0x557f437872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377d440 0x557f43786d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 0x557f437872d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38206/0 (socket says 172.21.15.67:38206) 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 -- 172.21.15.67:0/3538714167 learned_addr learned my addr 172.21.15.67:0/3538714167 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63377fe700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377ecf0 0x557f4378e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 -- 172.21.15.67:0/3538714167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377ecf0 msgr2=0x557f4378e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377ecf0 0x557f4378e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 -- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377d440 msgr2=0x557f43786d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377d440 0x557f43786d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 -- 172.21.15.67:0/3538714167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633811c6a0 con 0x557f4377dd90 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f6336ffd700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377d440 0x557f43786d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f63367fc700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 0x557f437872d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f632c00aee0 tx=0x7f632c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa2254fcbf933b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f631a7f4700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f631a7f4700 1 -- 172.21.15.67:0/3538714167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00e070 con 0x557f4377dd90 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f631a7f4700 1 -- 172.21.15.67:0/3538714167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f632c0090d0 con 0x557f4377dd90 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.367+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f43770ce0 con 0x557f4377dd90 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f631a7f4700 1 -- 172.21.15.67:0/3538714167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f632c0094a0 con 0x557f4377dd90 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 10 client.15219 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f43508fe0 con 0x557f4377dd90 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f631a7f4700 1 -- 172.21.15.67:0/3538714167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f43770ce0 con 0x557f4377dd90 2023-10-07T23:17:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f631a7f4700 1 -- 172.21.15.67:0/3538714167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f632c025020 con 0x557f4377dd90 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 10 client.15219 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 10 client.15219 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 20 client.15219 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f43791410 0x557f437971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 4 client.15219 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.368+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 --> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f4374dab0 con 0x557f43791410 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.369+0000 7f6336ffd700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f43791410 0x557f437971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.369+0000 7f6336ffd700 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f43791410 0x557f437971f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb70f943fe39adb5 server_cookie=5cd07a8782f82778 in_seq=0 out_seq=0 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.369+0000 7f631a7f4700 10 client.15219 ms_handle_connect on v2:172.21.15.86:6836/4272295866 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f631a7f4700 1 -- 172.21.15.67:0/3538714167 <== mds.2 v2:172.21.15.86:6836/4272295866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557f4374dab0 con 0x557f43791410 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f631a7f4700 10 client.15219 handle_command_reply: tid=0 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 client.15219 shutdown 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 msgr2=0x557f437872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 0x557f437872d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f632c00aee0 tx=0x7f632c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 shutdown_connections 2023-10-07T23:17:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f43791410 0x557f437971f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377d440 0x557f43786d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377ecf0 0x557f4378e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 --2- 172.21.15.67:0/3538714167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377dd90 0x557f437872d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f631a7f4700 0 client.15219 ms_handle_reset on v2:172.21.15.86:6836/4272295866 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 >> 172.21.15.67:0/3538714167 conn(0x557f43778470 msgr2=0x557f43772f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 shutdown_connections 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.370+0000 7f634bddab80 1 -- 172.21.15.67:0/3538714167 wait complete. 2023-10-07T23:17:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.371+0000 7f634bddab80 20 client.15219 trim_cache size 0 max 16384 2023-10-07T23:17:17.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 Processor -- start 2023-10-07T23:17:17.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 -- start start 2023-10-07T23:17:17.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 0x557f43785370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377dd90 0x557f43779480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377ecf0 0x557f437799c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43697d90 con 0x557f4377d440 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f436782e0 con 0x557f4377ecf0 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f634bddab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43564030 con 0x557f4377dd90 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 0x557f43785370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 0x557f43785370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38214/0 (socket says 172.21.15.67:38214) 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 -- 172.21.15.67:0/477768875 learned_addr learned my addr 172.21.15.67:0/477768875 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f63377fe700 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377ecf0 0x557f437799c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f63367fc700 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377dd90 0x557f43779480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 -- 172.21.15.67:0/477768875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377dd90 msgr2=0x557f43779480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377dd90 0x557f43779480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 -- 172.21.15.67:0/477768875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377ecf0 msgr2=0x557f437799c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377ecf0 0x557f437799c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f6336ffd700 1 -- 172.21.15.67:0/477768875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4369a780 con 0x557f4377d440 2023-10-07T23:17:17.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.388+0000 7f63367fc700 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377dd90 0x557f43779480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:17.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.389+0000 7f6336ffd700 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 0x557f43785370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6328093ad0 tx=0x7f63280925b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7b5a10c4bf2c38 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.389+0000 7f631d7fa700 1 -- 172.21.15.67:0/477768875 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280aa040 con 0x557f4377d440 2023-10-07T23:17:17.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.389+0000 7f631d7fa700 1 -- 172.21.15.67:0/477768875 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6328093b40 con 0x557f4377d440 2023-10-07T23:17:17.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.389+0000 7f631d7fa700 1 -- 172.21.15.67:0/477768875 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632809b030 con 0x557f4377d440 2023-10-07T23:17:17.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:17 smithi123 ceph-mon[122926]: pgmap v345: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:17.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:17 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3964154068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:17.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:17 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1647813621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:17.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.389+0000 7f634bddab80 1 -- 172.21.15.67:0/477768875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 msgr2=0x557f43785370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.389+0000 7f634bddab80 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 0x557f43785370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6328093ad0 tx=0x7f63280925b0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 -- 172.21.15.67:0/477768875 shutdown_connections 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377ecf0 0x557f437799c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377d440 0x557f43785370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 --2- 172.21.15.67:0/477768875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377dd90 0x557f43779480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 -- 172.21.15.67:0/477768875 >> 172.21.15.67:0/477768875 conn(0x557f43778470 msgr2=0x557f43784c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 -- 172.21.15.67:0/477768875 shutdown_connections 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 -- 172.21.15.67:0/477768875 wait complete. 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 -- start start 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.390+0000 7f634bddab80 1 Processor -- start 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f43783fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f437e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f437e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f437e4890 con 0x557f4377ecf0 2023-10-07T23:17:17.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f437e49d0 con 0x557f4377dd90 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f437e4b10 con 0x557f4377d440 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f634bddab80 10 client.0 _ll_register_callbacks cb 0x557f4373f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f437e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f437e4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38216/0 (socket says 172.21.15.67:38216) 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 -- 172.21.15.67:0/2471021497 learned_addr learned my addr 172.21.15.67:0/2471021497 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63367fc700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f437e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f6336ffd700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f43783fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 -- 172.21.15.67:0/2471021497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 msgr2=0x557f43783fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f43783fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 -- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 msgr2=0x557f437e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f437e1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63377fe700 1 -- 172.21.15.67:0/2471021497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f43566270 con 0x557f4377ecf0 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.391+0000 7f63367fc700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f437e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f63377fe700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f437e4350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f63300097f0 tx=0x7f6330009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfa989606858fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f631a7f4700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f631a7f4700 1 -- 172.21.15.67:0/2471021497 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000e040 con 0x557f4377ecf0 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6328093960 con 0x557f4377ecf0 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f631a7f4700 1 -- 172.21.15.67:0/2471021497 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f6330019e10 con 0x557f4377ecf0 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f631a7f4700 1 -- 172.21.15.67:0/2471021497 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6330009820 con 0x557f4377ecf0 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f634bddab80 10 client.15225 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f43681470 con 0x557f4377ecf0 2023-10-07T23:17:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.392+0000 7f631a7f4700 1 -- 172.21.15.67:0/2471021497 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6328093960 con 0x557f4377ecf0 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f631a7f4700 1 -- 172.21.15.67:0/2471021497 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f6330024050 con 0x557f4377ecf0 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f634bddab80 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f634bddab80 10 client.15225 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f634bddab80 20 client.15225 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f634bddab80 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f437e97e0 0x557f437e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f634bddab80 4 client.15225 mds_command: new command op to 24521 tid=0[{"prefix": "status"}] 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 --> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f4366ee00 con 0x557f437e97e0 2023-10-07T23:17:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f6336ffd700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f437e97e0 0x557f437e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f631a7f4700 10 client.15225 ms_handle_connect on v2:172.21.15.86:6836/4272295866 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.393+0000 7f6336ffd700 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f437e97e0 0x557f437e6150 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=236fafe4654b2b8d server_cookie=ed784bd2b3966811 in_seq=0 out_seq=0 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f631a7f4700 1 -- 172.21.15.67:0/2471021497 <== mds.2 v2:172.21.15.86:6836/4272295866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557f4366ee00 con 0x557f437e97e0 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f631a7f4700 10 client.15225 handle_command_reply: tid=0 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 client.15225 shutdown 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 msgr2=0x557f437e4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f437e4350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f63300097f0 tx=0x7f6330009000 comp rx=0 tx=0).stop 2023-10-07T23:17:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 shutdown_connections 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:6836/4272295866,v1:172.21.15.86:6837/4272295866] conn(0x557f437e97e0 0x557f437e6150 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f4377ecf0 0x557f437e4350 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f4377d440 0x557f43783fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f631a7f4700 0 client.15225 ms_handle_reset on v2:172.21.15.86:6836/4272295866 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 --2- 172.21.15.67:0/2471021497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f4377dd90 0x557f437e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 >> 172.21.15.67:0/2471021497 conn(0x557f43778470 msgr2=0x557f43772670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 shutdown_connections 2023-10-07T23:17:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 1 -- 172.21.15.67:0/2471021497 wait complete. 2023-10-07T23:17:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.394+0000 7f634bddab80 20 client.15225 trim_cache size 0 max 16384 2023-10-07T23:17:17.407 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:17:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:17:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2023-10-07T23:17:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24521, 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 84.382629694000002, 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:17:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:17:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 128.328851786 2023-10-07T23:17:17.409 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f632009b7f0 msgr2=0x7f632009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f632009b7f0 0x7f632009dca0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63300049c0 tx=0x7f633000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 msgr2=0x7f6338100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f6338100020 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f632800ebf0 tx=0x7f632800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 shutdown_connections 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6338070570 0x7f6338100020 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f632009b7f0 0x7f632009dca0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6338104b80 0x7f63380fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 --2- 172.21.15.67:0/1374772991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338071bc0 0x7f6338100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 >> 172.21.15.67:0/1374772991 conn(0x7f63380f8460 msgr2=0x7f63380fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 shutdown_connections 2023-10-07T23:17:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.395+0000 7f6337fff700 1 -- 172.21.15.67:0/1374772991 wait complete. 2023-10-07T23:17:17.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-07T23:17:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:17 smithi086 ceph-mon[122131]: pgmap v345: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:17 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3964154068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:17.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:17 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1647813621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:17.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 -- 172.21.15.67:0/1692230221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea4100aa0 msgr2=0x7f8ea41073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1692230221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea4100aa0 0x7f8ea41073d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c004ca0 tx=0x7f8e9c009ca0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 -- 172.21.15.67:0/1692230221 shutdown_connections 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1692230221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea40fef00 0x7f8ea40ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1692230221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea4100aa0 0x7f8ea41073d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1692230221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ea41000f0 0x7f8ea4100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 -- 172.21.15.67:0/1692230221 >> 172.21.15.67:0/1692230221 conn(0x7f8ea40fa790 msgr2=0x7f8ea40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 -- 172.21.15.67:0/1692230221 shutdown_connections 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.602+0000 7f8ea359e700 1 -- 172.21.15.67:0/1692230221 wait complete. 2023-10-07T23:17:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.603+0000 7f8ea359e700 1 Processor -- start 2023-10-07T23:17:17.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.603+0000 7f8ea359e700 1 -- start start 2023-10-07T23:17:17.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.603+0000 7f8ea359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 0x7f8ea410fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.603+0000 7f8ea359e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea41000f0 0x7f8ea4110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ea4100aa0 0x7f8ea41170d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea359e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41176e0 con 0x7f8ea40fef00 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea359e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4117850 con 0x7f8ea41000f0 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea359e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4117a10 con 0x7f8ea4100aa0 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 0x7f8ea410fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 0x7f8ea410fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38224/0 (socket says 172.21.15.67:38224) 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 -- 172.21.15.67:0/1808224431 learned_addr learned my addr 172.21.15.67:0/1808224431 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ea4100aa0 0x7f8ea41170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea41000f0 0x7f8ea4110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 -- 172.21.15.67:0/1808224431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ea4100aa0 msgr2=0x7f8ea41170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ea4100aa0 0x7f8ea41170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 -- 172.21.15.67:0/1808224431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea41000f0 msgr2=0x7f8ea4110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea41000f0 0x7f8ea4110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 -- 172.21.15.67:0/1808224431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0048f0 con 0x7f8ea40fef00 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.604+0000 7f8ea259c700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 0x7f8ea410fb00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400ebf0 tx=0x7f8e9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.605+0000 7f8e92ffd700 1 -- 172.21.15.67:0/1808224431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94007cc0 con 0x7f8ea40fef00 2023-10-07T23:17:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.605+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea4117c90 con 0x7f8ea40fef00 2023-10-07T23:17:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.605+0000 7f8e92ffd700 1 -- 172.21.15.67:0/1808224431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e9400f040 con 0x7f8ea40fef00 2023-10-07T23:17:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.605+0000 7f8e92ffd700 1 -- 172.21.15.67:0/1808224431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400c8f0 con 0x7f8ea40fef00 2023-10-07T23:17:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.605+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea4118180 con 0x7f8ea40fef00 2023-10-07T23:17:17.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.608+0000 7f8e92ffd700 1 -- 172.21.15.67:0/1808224431 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f8e9400cd20 con 0x7f8ea40fef00 2023-10-07T23:17:17.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.609+0000 7f8eb7118b80 1 Processor -- start 2023-10-07T23:17:17.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.609+0000 7f8eb7118b80 1 -- start start 2023-10-07T23:17:17.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.609+0000 7f8e92ffd700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8e8c09b770 0x7f8e8c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.609+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada6b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8e92ffd700 1 -- 172.21.15.67:0/1808224431 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f8e940c4ca0 con 0x7f8ea40fef00 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ad9c0a70 con 0x5593ada6ccf0 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ad9be270 con 0x5593ada6bd90 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ad9ba060 con 0x5593ada6b440 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8e8c09b770 0x7f8e8c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada6b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada6d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38228/0 (socket says 172.21.15.67:38228) 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/3421558740 learned_addr learned my addr 172.21.15.67:0/3421558740 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea259c700 1 -- 172.21.15.67:0/3421558740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 msgr2=0x5593ada6c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea259c700 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada6c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea259c700 1 -- 172.21.15.67:0/3421558740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 msgr2=0x5593ada6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea259c700 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.610+0000 7f8ea259c700 1 -- 172.21.15.67:0/3421558740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593ad854270 con 0x5593ada6b440 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8e8c09b770 0x7f8e8c09dc20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c004860 tx=0x7f8e9c009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8ea259c700 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada6b850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8e940c2430 tx=0x7f8e940c2300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=304e1289ea0eaebd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8e8bfff700 1 -- 172.21.15.67:0/3421558740 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94007e20 con 0x5593ada6b440 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8e8bfff700 1 -- 172.21.15.67:0/3421558740 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e94094510 con 0x5593ada6b440 2023-10-07T23:17:17.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8e8bfff700 1 -- 172.21.15.67:0/3421558740 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94091fe0 con 0x5593ada6b440 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8eb7118b80 1 -- 172.21.15.67:0/3421558740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 msgr2=0x5593ada6b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.611+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada6b850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8e940c2430 tx=0x7f8e940c2300 comp rx=0 tx=0).stop 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 -- 172.21.15.67:0/3421558740 shutdown_connections 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada6b850 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/3421558740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 -- 172.21.15.67:0/3421558740 >> 172.21.15.67:0/3421558740 conn(0x5593ada66470 msgr2=0x5593ada66860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 -- 172.21.15.67:0/3421558740 shutdown_connections 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 -- 172.21.15.67:0/3421558740 wait complete. 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 -- start start 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.612+0000 7f8eb7118b80 1 Processor -- start 2023-10-07T23:17:17.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada74d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada75260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ad965890 con 0x5593ada6ccf0 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ad980ea0 con 0x5593ada6bd90 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ad83ce30 con 0x5593ada6b440 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8eb7118b80 10 client.0 _ll_register_callbacks cb 0x5593ada2d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8ea1d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada75260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8ea1d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada75260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39826/0 (socket says 172.21.15.67:39826) 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8ea1d9b700 1 -- 172.21.15.67:0/4037518287 learned_addr learned my addr 172.21.15.67:0/4037518287 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.613+0000 7f8ea259c700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada74d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 msgr2=0x5593ada74d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada74d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/4037518287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 msgr2=0x5593ada75260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada75260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/4037518287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea4118180 con 0x5593ada6ccf0 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada75260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea259c700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada74d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada7c2c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005010 tx=0x7f8e9c003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f884fc22e37a9c2c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8e897fa700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8e897fa700 1 -- 172.21.15.67:0/4037518287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c030070 con 0x5593ada6ccf0 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8e897fa700 1 -- 172.21.15.67:0/4037518287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f8e9c022440 con 0x5593ada6ccf0 2023-10-07T23:17:17.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593ada5ece0 con 0x5593ada6ccf0 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8e897fa700 1 -- 172.21.15.67:0/4037518287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e9c027070 con 0x5593ada6ccf0 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8eb7118b80 10 client.15237 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.614+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5593ad7f6fe0 con 0x5593ada6ccf0 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8e897fa700 1 -- 172.21.15.67:0/4037518287 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593ada5ece0 con 0x5593ada6ccf0 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8e897fa700 1 -- 172.21.15.67:0/4037518287 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f8e9c033030 con 0x5593ada6ccf0 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8eb7118b80 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8eb7118b80 10 client.15237 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8eb7118b80 20 client.15237 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593ada7fd00 0x5593ada85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8eb7118b80 4 client.15237 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:17:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 --> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593ada3bab0 con 0x5593ada7fd00 2023-10-07T23:17:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.615+0000 7f8ea259c700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593ada7fd00 0x5593ada85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.616+0000 7f8ea259c700 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593ada7fd00 0x5593ada85020 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e69e84d8c7d0eec6 server_cookie=d2270840b8850dd8 in_seq=0 out_seq=0 2023-10-07T23:17:17.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.616+0000 7f8e897fa700 10 client.15237 ms_handle_connect on v2:172.21.15.123:6838/3252673889 2023-10-07T23:17:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8e897fa700 1 -- 172.21.15.67:0/4037518287 <== mds.1 v2:172.21.15.123:6838/3252673889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5593ada3bab0 con 0x5593ada7fd00 2023-10-07T23:17:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8e897fa700 10 client.15237 handle_command_reply: tid=0 2023-10-07T23:17:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 client.15237 shutdown 2023-10-07T23:17:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 msgr2=0x5593ada7c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada7c2c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005010 tx=0x7f8e9c003000 comp rx=0 tx=0).stop 2023-10-07T23:17:17.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 shutdown_connections 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593ada7fd00 0x5593ada85020 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada7c2c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6b440 0x5593ada74d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8e897fa700 0 client.15237 ms_handle_reset on v2:172.21.15.123:6838/3252673889 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/4037518287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6bd90 0x5593ada75260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 >> 172.21.15.67:0/4037518287 conn(0x5593ada66470 msgr2=0x5593ada67ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 shutdown_connections 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 1 -- 172.21.15.67:0/4037518287 wait complete. 2023-10-07T23:17:17.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.617+0000 7f8eb7118b80 20 client.15237 trim_cache size 0 max 16384 2023-10-07T23:17:17.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.634+0000 7f8eb7118b80 1 Processor -- start 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 -- start start 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada6f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6bd90 0x5593ada6fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada72bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ada6c3b0 con 0x5593ada6ccf0 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ada3bab0 con 0x5593ada6b440 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ada6c560 con 0x5593ada6bd90 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada72bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada72bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38236/0 (socket says 172.21.15.67:38236) 2023-10-07T23:17:17.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/2206085214 learned_addr learned my addr 172.21.15.67:0/2206085214 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8ea259c700 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada6f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.635+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6bd90 0x5593ada6fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/2206085214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6bd90 msgr2=0x5593ada6fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6bd90 0x5593ada6fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/2206085214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 msgr2=0x5593ada6f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada6f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea2d9d700 1 -- 172.21.15.67:0/2206085214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593ad988780 con 0x5593ada6ccf0 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea259c700 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada6f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada72bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00bf80 tx=0x7f8e9c024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fce2cc6f29f3d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8e90ff9700 1 -- 172.21.15.67:0/2206085214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c041070 con 0x5593ada6ccf0 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8e90ff9700 1 -- 172.21.15.67:0/2206085214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e9c0043f0 con 0x5593ada6ccf0 2023-10-07T23:17:17.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.636+0000 7f8e90ff9700 1 -- 172.21.15.67:0/2206085214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c027070 con 0x5593ada6ccf0 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 -- 172.21.15.67:0/2206085214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 msgr2=0x5593ada72bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada72bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00bf80 tx=0x7f8e9c024040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 -- 172.21.15.67:0/2206085214 shutdown_connections 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada6f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6ccf0 0x5593ada72bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/2206085214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6bd90 0x5593ada6fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 -- 172.21.15.67:0/2206085214 >> 172.21.15.67:0/2206085214 conn(0x5593ada66470 msgr2=0x5593ada66840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 -- 172.21.15.67:0/2206085214 shutdown_connections 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 -- 172.21.15.67:0/2206085214 wait complete. 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 -- start start 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.637+0000 7f8eb7118b80 1 Processor -- start 2023-10-07T23:17:17.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 0x5593ada78d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6ccf0 0x5593ada79280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adad1c40 con 0x5593ada6bd90 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adad1d80 con 0x5593ada6b440 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adad1f10 con 0x5593ada6ccf0 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8eb7118b80 10 client.0 _ll_register_callbacks cb 0x5593ada2d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 0x5593ada78d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 0x5593ada78d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38244/0 (socket says 172.21.15.67:38244) 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 -- 172.21.15.67:0/865346121 learned_addr learned my addr 172.21.15.67:0/865346121 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea259c700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6ccf0 0x5593ada79280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 -- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6ccf0 msgr2=0x5593ada79280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6ccf0 0x5593ada79280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 -- 172.21.15.67:0/865346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 msgr2=0x5593ada78800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada78800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea1d9b700 1 -- 172.21.15.67:0/865346121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593ad7f6fe0 con 0x5593ada6bd90 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.638+0000 7f8ea2d9d700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6ccf0 0x5593ada79280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8ea1d9b700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 0x5593ada78d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e98016480 tx=0x7f8e98002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4485c96a44f93b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8e897fa700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8e897fa700 1 -- 172.21.15.67:0/865346121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801b070 con 0x5593ada6bd90 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e94094510 con 0x5593ada6bd90 2023-10-07T23:17:17.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8e897fa700 1 -- 172.21.15.67:0/865346121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f8e98007cc0 con 0x5593ada6bd90 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8e897fa700 1 -- 172.21.15.67:0/865346121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e9800e040 con 0x5593ada6bd90 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8eb7118b80 10 client.15243 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5593ad96f470 con 0x5593ada6bd90 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8e897fa700 1 -- 172.21.15.67:0/865346121 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e94094510 con 0x5593ada6bd90 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8e897fa700 1 -- 172.21.15.67:0/865346121 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f8e98020990 con 0x5593ada6bd90 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.639+0000 7f8eb7118b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8eb7118b80 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8eb7118b80 20 client.15243 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593adad6ce0 0x5593adad3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8eb7118b80 4 client.15243 mds_command: new command op to 24538 tid=0[{"prefix": "status"}] 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 --> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5593ad95ce00 con 0x5593adad6ce0 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8ea259c700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593adad6ce0 0x5593adad3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8ea259c700 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593adad6ce0 0x5593adad3610 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40afe01b3285e1c1 server_cookie=6579e11f336fa04b in_seq=0 out_seq=0 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8e897fa700 10 client.15243 ms_handle_connect on v2:172.21.15.123:6838/3252673889 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8e897fa700 1 -- 172.21.15.67:0/865346121 <== mds.1 v2:172.21.15.123:6838/3252673889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5593ad95ce00 con 0x5593adad6ce0 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.640+0000 7f8e897fa700 10 client.15243 handle_command_reply: tid=0 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 client.15243 shutdown 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 msgr2=0x5593ada78d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 0x5593ada78d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e98016480 tx=0x7f8e98002040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 shutdown_connections 2023-10-07T23:17:17.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:6838/3252673889,v1:172.21.15.123:6839/3252673889] conn(0x5593adad6ce0 0x5593adad3610 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593ada6b440 0x5593ada78800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593ada6ccf0 0x5593ada79280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 --2- 172.21.15.67:0/865346121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5593ada6bd90 0x5593ada78d40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8e897fa700 0 client.15243 ms_handle_reset on v2:172.21.15.123:6838/3252673889 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 >> 172.21.15.67:0/865346121 conn(0x5593ada66470 msgr2=0x5593ada6f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 shutdown_connections 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 1 -- 172.21.15.67:0/865346121 wait complete. 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.641+0000 7f8eb7118b80 20 client.15243 trim_cache size 0 max 16384 2023-10-07T23:17:17.648 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24538, 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 85.633771744000001, 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:17:17.649 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 126.010746169 2023-10-07T23:17:17.650 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:17:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8e8c09b770 msgr2=0x7f8e8c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8e8c09b770 0x7f8e8c09dc20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c004860 tx=0x7f8e9c009ca0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 msgr2=0x7f8ea410fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 0x7f8ea410fb00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400ebf0 tx=0x7f8e9400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 shutdown_connections 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f8e8c09b770 0x7f8e8c09dc20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ea4100aa0 0x7f8ea41170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ea40fef00 0x7f8ea410fb00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 --2- 172.21.15.67:0/1808224431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea41000f0 0x7f8ea4110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 >> 172.21.15.67:0/1808224431 conn(0x7f8ea40fa790 msgr2=0x7f8ea40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 shutdown_connections 2023-10-07T23:17:17.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.642+0000 7f8ea359e700 1 -- 172.21.15.67:0/1808224431 wait complete. 2023-10-07T23:17:17.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 -- 172.21.15.67:0/2829177125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 msgr2=0x7f39d80fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/2829177125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d80fe340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0049c0 tx=0x7f39cc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 -- 172.21.15.67:0/2829177125 shutdown_connections 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/2829177125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d80fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/2829177125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 0x7f39d8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/2829177125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d80fe340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 -- 172.21.15.67:0/2829177125 >> 172.21.15.67:0/2829177125 conn(0x7f39d80f8460 msgr2=0x7f39d80fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 -- 172.21.15.67:0/2829177125 shutdown_connections 2023-10-07T23:17:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.849+0000 7f39dc9fe700 1 -- 172.21.15.67:0/2829177125 wait complete. 2023-10-07T23:17:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 Processor -- start 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 -- start start 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 0x7f39d8114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d81155d0 con 0x7f39d80fdef0 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8115740 con 0x7f39d80fcce0 2023-10-07T23:17:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39dc9fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8115900 con 0x7f39d80fe880 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39d659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39866/0 (socket says 172.21.15.67:39866) 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1751545290 learned_addr learned my addr 172.21.15.67:0/1751545290 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.850+0000 7f39d759e700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 0x7f39d8114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1751545290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 msgr2=0x7f39d8114fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 0x7f39d8114fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 msgr2=0x7f39d810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1751545290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc005040 con 0x7f39d80fcce0 2023-10-07T23:17:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d759e700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 0x7f39d8114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d659c700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d810d9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39c800ebf0 tx=0x7f39c800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39bf7fe700 1 -- 172.21.15.67:0/1751545290 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8007cc0 con 0x7f39d80fcce0 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d8115b80 con 0x7f39d80fcce0 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39bf7fe700 1 -- 172.21.15.67:0/1751545290 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39c800f040 con 0x7f39d80fcce0 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.851+0000 7f39bf7fe700 1 -- 172.21.15.67:0/1751545290 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800c8f0 con 0x7f39d80fcce0 2023-10-07T23:17:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.852+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d81160c0 con 0x7f39d80fcce0 2023-10-07T23:17:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.855+0000 7f39bf7fe700 1 -- 172.21.15.67:0/1751545290 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f39c800cd20 con 0x7f39d80fcce0 2023-10-07T23:17:17.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 Processor -- start 2023-10-07T23:17:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 -- start start 2023-10-07T23:17:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76e3b0 0x55f95f76e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f76f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76fcb0 0x55f95f770120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39bf7fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f39c009b770 0x7f39c009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f6c3a70 con 0x55f95f76ed00 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f6c1270 con 0x55f95f76fcb0 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f6bd060 con 0x55f95f76e3b0 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39bf7fe700 1 -- 172.21.15.67:0/1751545290 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f39c80c4ca0 con 0x7f39d80fcce0 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f76f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39d759e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76fcb0 0x55f95f770120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f76f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38280/0 (socket says 172.21.15.67:38280) 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1179943232 learned_addr learned my addr 172.21.15.67:0/1179943232 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39d659c700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f39c009b770 0x7f39c009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.856+0000 7f39d759e700 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76e3b0 0x55f95f76e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1179943232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76e3b0 msgr2=0x55f95f76e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76e3b0 0x55f95f76e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1179943232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76fcb0 msgr2=0x55f95f770120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76fcb0 0x55f95f770120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1179943232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f95f557270 con 0x55f95f76ed00 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d759e700 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76fcb0 0x55f95f770120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f76f690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39c80c2430 tx=0x7f39c80c2300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10124399fddfaaa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39d659c700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f39c009b770 0x7f39c009dc20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39cc004890 tx=0x7f39cc009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39bcff9700 1 -- 172.21.15.67:0/1179943232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8007e20 con 0x55f95f76ed00 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39bcff9700 1 -- 172.21.15.67:0/1179943232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39c8094510 con 0x55f95f76ed00 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.857+0000 7f39bcff9700 1 -- 172.21.15.67:0/1179943232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c80922d0 con 0x55f95f76ed00 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1179943232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 msgr2=0x55f95f76f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f76f690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39c80c2430 tx=0x7f39c80c2300 comp rx=0 tx=0).stop 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1179943232 shutdown_connections 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76fcb0 0x55f95f770120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76e3b0 0x55f95f76e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1179943232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f76f690 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1179943232 >> 172.21.15.67:0/1179943232 conn(0x55f95f769470 msgr2=0x55f95f769860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1179943232 shutdown_connections 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.858+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1179943232 wait complete. 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 -- start start 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 Processor -- start 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f7806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f780c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76fcb0 0x55f95f787c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f668890 con 0x55f95f76ed00 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f683ea0 con 0x55f95f76e3b0 2023-10-07T23:17:17.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f53fe30 con 0x55f95f76fcb0 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.859+0000 7f39eb8d9b80 10 client.0 _ll_register_callbacks cb 0x55f95f7302c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f780c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f780c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38296/0 (socket says 172.21.15.67:38296) 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 -- 172.21.15.67:0/3169682000 learned_addr learned my addr 172.21.15.67:0/3169682000 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d759e700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76fcb0 0x55f95f787c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f7806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 -- 172.21.15.67:0/3169682000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76fcb0 msgr2=0x55f95f787c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76fcb0 0x55f95f787c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 -- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 msgr2=0x55f95f7806c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f7806c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 -- 172.21.15.67:0/3169682000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d0002c50 con 0x55f95f76ed00 2023-10-07T23:17:17.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39d659c700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f780c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f39cc013000 tx=0x7f39cc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ae366a2a34f4a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.860+0000 7f39ba7f4700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39ba7f4700 1 -- 172.21.15.67:0/3169682000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc01eae0 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39ba7f4700 1 -- 172.21.15.67:0/3169682000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f39cc023030 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f95f761ce0 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39ba7f4700 1 -- 172.21.15.67:0/3169682000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39cc01ee50 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39eb8d9b80 10 client.15255 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f39d81160c0 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39ba7f4700 1 -- 172.21.15.67:0/3169682000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f95f761ce0 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39ba7f4700 1 -- 172.21.15.67:0/3169682000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f39cc01d3e0 con 0x55f95f76ed00 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.861+0000 7f39eb8d9b80 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39eb8d9b80 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39eb8d9b80 20 client.15255 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f78b870 0x55f95f790b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39eb8d9b80 4 client.15255 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 --> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f95f73eab0 con 0x55f95f78b870 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f78b870 0x55f95f790b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f78b870 0x55f95f790b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819d0920d3129563 server_cookie=fa5829c67ae079ce in_seq=0 out_seq=0 2023-10-07T23:17:17.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.862+0000 7f39ba7f4700 10 client.15255 ms_handle_connect on v2:172.21.15.86:6838/2147538694 2023-10-07T23:17:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39ba7f4700 1 -- 172.21.15.67:0/3169682000 <== mds.0 v2:172.21.15.86:6838/2147538694 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f95f73eab0 con 0x55f95f78b870 2023-10-07T23:17:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39ba7f4700 10 client.15255 handle_command_reply: tid=0 2023-10-07T23:17:17.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 client.15255 shutdown 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 msgr2=0x55f95f780c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f780c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f39cc013000 tx=0x7f39cc020000 comp rx=0 tx=0).stop 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 shutdown_connections 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f78b870 0x55f95f790b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f7806c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76fcb0 0x55f95f787c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39ba7f4700 0 client.15255 ms_handle_reset on v2:172.21.15.86:6838/2147538694 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/3169682000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76ed00 0x55f95f780c00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 >> 172.21.15.67:0/3169682000 conn(0x55f95f769470 msgr2=0x55f95f763f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 shutdown_connections 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.863+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/3169682000 wait complete. 2023-10-07T23:17:17.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.864+0000 7f39eb8d9b80 20 client.15255 trim_cache size 0 max 16384 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 Processor -- start 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 -- start start 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f763670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f763bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f76a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f763500 con 0x55f95f76fcb0 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f6692b0 con 0x55f95f76e3b0 2023-10-07T23:17:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.881+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f581eb0 con 0x55f95f76ed00 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f763670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d759e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f76a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f763bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f763670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39894/0 (socket says 172.21.15.67:39894) 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1190003938 learned_addr learned my addr 172.21.15.67:0/1190003938 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1190003938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 msgr2=0x55f95f763bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f763bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1190003938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 msgr2=0x55f95f76a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f76a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 -- 172.21.15.67:0/1190003938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f95f4f9fe0 con 0x55f95f76e3b0 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f763670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f39c8000c00 tx=0x7f39c8096040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb116fe87f8fb20 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39bd7fa700 1 -- 172.21.15.67:0/1190003938 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c809d030 con 0x55f95f76e3b0 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39bd7fa700 1 -- 172.21.15.67:0/1190003938 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39c809ae40 con 0x55f95f76e3b0 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.882+0000 7f39bd7fa700 1 -- 172.21.15.67:0/1190003938 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8092220 con 0x55f95f76e3b0 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1190003938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 msgr2=0x55f95f763670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f763670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f39c8000c00 tx=0x7f39c8096040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1190003938 shutdown_connections 2023-10-07T23:17:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f763670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f76a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/1190003938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f763bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1190003938 >> 172.21.15.67:0/1190003938 conn(0x55f95f769470 msgr2=0x55f95f780660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1190003938 shutdown_connections 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/1190003938 wait complete. 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.883+0000 7f39eb8d9b80 1 -- start start 2023-10-07T23:17:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 Processor -- start 2023-10-07T23:17:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f77d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f77d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f7dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f7ddcb0 con 0x55f95f76fcb0 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f7dddf0 con 0x55f95f76e3b0 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95f7ddf30 con 0x55f95f76ed00 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39eb8d9b80 10 client.0 _ll_register_callbacks cb 0x55f95f7302c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f77d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39d759e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f7dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39d6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f77d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39898/0 (socket says 172.21.15.67:39898) 2023-10-07T23:17:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39d6d9d700 1 -- 172.21.15.67:0/2201374894 learned_addr learned my addr 172.21.15.67:0/2201374894 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.884+0000 7f39d659c700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f77d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d6d9d700 1 -- 172.21.15.67:0/2201374894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 msgr2=0x55f95f77d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f77d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d6d9d700 1 -- 172.21.15.67:0/2201374894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 msgr2=0x55f95f7dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f7dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d6d9d700 1 -- 172.21.15.67:0/2201374894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c80aa040 con 0x55f95f76e3b0 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d759e700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f7dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39d6d9d700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f77d320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39c8090310 tx=0x7f39c80c4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731f0135249636a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39ba7f4700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39ba7f4700 1 -- 172.21.15.67:0/2201374894 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c809d070 con 0x55f95f76e3b0 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39c8094510 con 0x55f95f76e3b0 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39ba7f4700 1 -- 172.21.15.67:0/2201374894 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f39c809e070 con 0x55f95f76e3b0 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39ba7f4700 1 -- 172.21.15.67:0/2201374894 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39c8093e10 con 0x55f95f76e3b0 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39eb8d9b80 10 client.25310 fetch_fsmap learned FSMap version 28 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f95f68b780 con 0x55f95f76e3b0 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.885+0000 7f39ba7f4700 1 -- 172.21.15.67:0/2201374894 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39c8094510 con 0x55f95f76e3b0 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39ba7f4700 1 -- 172.21.15.67:0/2201374894 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7456+0+0 (secure 0 0 0) 0x7f39c80a6100 con 0x55f95f76e3b0 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39eb8d9b80 10 client.25310 fetch_fsmap finished waiting for FSMap version 28 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39eb8d9b80 10 client.25310 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39eb8d9b80 20 client.25310 populate_metadata read hostname 'smithi067' 2023-10-07T23:17:17.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f7e26c0 0x55f95f7df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39eb8d9b80 4 client.25310 mds_command: new command op to 24563 tid=0[{"prefix": "status"}] 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 --> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f95f65fe00 con 0x55f95f7e26c0 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39d659c700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f7e26c0 0x55f95f7df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39d659c700 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f7e26c0 0x55f95f7df0f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125e60971ab58524 server_cookie=dc12206ee3d99f50 in_seq=0 out_seq=0 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.886+0000 7f39ba7f4700 10 client.25310 ms_handle_connect on v2:172.21.15.86:6838/2147538694 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39ba7f4700 1 -- 172.21.15.67:0/2201374894 <== mds.0 v2:172.21.15.86:6838/2147538694 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f95f65fe00 con 0x55f95f7e26c0 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39ba7f4700 10 client.25310 handle_command_reply: tid=0 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 client.25310 shutdown 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 msgr2=0x55f95f77d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f77d320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39c8090310 tx=0x7f39c80c4040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 shutdown_connections 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:6838/2147538694,v1:172.21.15.86:6839/2147538694] conn(0x55f95f7e26c0 0x55f95f7df0f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95f76e3b0 0x55f95f77d320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f95f76fcb0 0x55f95f7dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 --2- 172.21.15.67:0/2201374894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f95f76ed00 0x55f95f77d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39ba7f4700 0 client.25310 ms_handle_reset on v2:172.21.15.86:6838/2147538694 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 >> 172.21.15.67:0/2201374894 conn(0x55f95f769470 msgr2=0x55f95f763670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 shutdown_connections 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 1 -- 172.21.15.67:0/2201374894 wait complete. 2023-10-07T23:17:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.887+0000 7f39eb8d9b80 20 client.25310 trim_cache size 0 max 16384 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "227de690-6566-11ee-8db5-212e2dc638e7", 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24563, 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 110.54954317799999, 2023-10-07T23:17:17.895 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2023-10-07T23:17:17.896 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2023-10-07T23:17:17.896 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2023-10-07T23:17:17.896 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 117.431200309 2023-10-07T23:17:17.896 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-07T23:17:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f39c009b770 msgr2=0x7f39c009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f39c009b770 0x7f39c009dc20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39cc004890 tx=0x7f39cc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 msgr2=0x7f39d810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d810d9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39c800ebf0 tx=0x7f39c800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 shutdown_connections 2023-10-07T23:17:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39d80fcce0 0x7f39d810d9f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f39c009b770 0x7f39c009dc20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d80fe880 0x7f39d8114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 --2- 172.21.15.67:0/1751545290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39d80fdef0 0x7f39d810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 >> 172.21.15.67:0/1751545290 conn(0x7f39d80f8460 msgr2=0x7f39d80fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 shutdown_connections 2023-10-07T23:17:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:17.888+0000 7f39dc9fe700 1 -- 172.21.15.67:0/1751545290 wait complete. 2023-10-07T23:17:17.906 INFO:teuthology.run_tasks:Running task kclient... 2023-10-07T23:17:17.920 INFO:tasks.kclient:Mounting kernel clients... 2023-10-07T23:17:17.920 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-07T23:17:17.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:17.921 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-07T23:17:17.958 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-07T23:17:17.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:17.959 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-07T23:17:18.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:18.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-10-07T23:17:18.155 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2'} 2023-10-07T23:17:18.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:08cac0187919fc3a247efdf445f652e2fb30a647 shell --fsid 227de690-6566-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-07T23:17:18.931 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-07T23:17:19.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:19 smithi067 ceph-mon[82723]: pgmap v346: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:17:19.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:19 smithi123 ceph-mon[122926]: pgmap v346: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:17:19.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:19 smithi086 ceph-mon[122131]: pgmap v346: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 -- 172.21.15.67:0/3165603691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0102e60 msgr2=0x7fd8a0103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 --2- 172.21.15.67:0/3165603691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0102e60 0x7fd8a0103230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd8900049f0 tx=0x7fd890009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 -- 172.21.15.67:0/3165603691 shutdown_connections 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 --2- 172.21.15.67:0/3165603691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a00fd550 0x7fd8a0109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 --2- 172.21.15.67:0/3165603691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0102e60 0x7fd8a0103230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 --2- 172.21.15.67:0/3165603691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a0103800 0x7fd8a00fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 -- 172.21.15.67:0/3165603691 >> 172.21.15.67:0/3165603691 conn(0x7fd8a0074320 msgr2=0x7fd8a0074720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 -- 172.21.15.67:0/3165603691 shutdown_connections 2023-10-07T23:17:20.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 -- 172.21.15.67:0/3165603691 wait complete. 2023-10-07T23:17:20.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.344+0000 7fd8a6458700 1 Processor -- start 2023-10-07T23:17:20.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 -- start start 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 0x7fd8a006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a0102e60 0x7fd8a0070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0103800 0x7fd8a00709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a011c1c0 con 0x7fd8a0103800 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a011c330 con 0x7fd8a00fd550 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a6458700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a011c4f0 con 0x7fd8a0102e60 2023-10-07T23:17:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd89ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 0x7fd8a006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd89ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 0x7fd8a006ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:39908/0 (socket says 172.21.15.67:39908) 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd89ffff700 1 -- 172.21.15.67:0/1841295695 learned_addr learned my addr 172.21.15.67:0/1841295695 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd8a49f5700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0103800 0x7fd8a00709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.345+0000 7fd89f7fe700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a0102e60 0x7fd8a0070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89ffff700 1 -- 172.21.15.67:0/1841295695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a0102e60 msgr2=0x7fd8a0070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89ffff700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a0102e60 0x7fd8a0070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89ffff700 1 -- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0103800 msgr2=0x7fd8a00709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89ffff700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0103800 0x7fd8a00709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89ffff700 1 -- 172.21.15.67:0/1841295695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd890005040 con 0x7fd8a00fd550 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89f7fe700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a0102e60 0x7fd8a0070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89ffff700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 0x7fd8a006ff40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd890009400 tx=0x7fd890009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000e070 con 0x7fd8a00fd550 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a011c770 con 0x7fd8a00fd550 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.346+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a011cc50 con 0x7fd8a00fd550 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.347+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8900099d0 con 0x7fd8a00fd550 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.347+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89001c430 con 0x7fd8a00fd550 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.347+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c0052e0 con 0x7fd8a00fd550 2023-10-07T23:17:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.349+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fd890026670 con 0x7fd8a00fd550 2023-10-07T23:17:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.351+0000 7fd89cff9700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd88809b7f0 0x7fd88809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.351+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fd8900c69c0 con 0x7fd8a00fd550 2023-10-07T23:17:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.351+0000 7fd89f7fe700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd88809b7f0 0x7fd88809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:20.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.352+0000 7fd89f7fe700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd88809b7f0 0x7fd88809dca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd894004600 tx=0x7fd89400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:20.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.353+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fd890091ef0 con 0x7fd8a00fd550 2023-10-07T23:17:20.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.602+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd88c005860 con 0x7fd8a00fd550 2023-10-07T23:17:20.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.603+0000 7fd89cff9700 1 -- 172.21.15.67:0/1841295695 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd890095510 con 0x7fd8a00fd550 2023-10-07T23:17:20.603 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-07T23:17:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd88809b7f0 msgr2=0x7fd88809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd88809b7f0 0x7fd88809dca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd894004600 tx=0x7fd89400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 msgr2=0x7fd8a006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 0x7fd8a006ff40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd890009400 tx=0x7fd890009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:20.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 shutdown_connections 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8a00fd550 0x7fd8a006ff40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fd88809b7f0 0x7fd88809dca0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0103800 0x7fd8a00709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 --2- 172.21.15.67:0/1841295695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8a0102e60 0x7fd8a0070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 >> 172.21.15.67:0/1841295695 conn(0x7fd8a0074320 msgr2=0x7fd8a00fb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 shutdown_connections 2023-10-07T23:17:20.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:20.605+0000 7fd8a6458700 1 -- 172.21.15.67:0/1841295695 wait complete. 2023-10-07T23:17:21.102 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:17:21.102 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:17:21.102 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2023-10-07T23:17:21.102 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T23:17:21.103 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T23:17:21.103 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:17:21.103 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:17:21.103 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:17:21.103 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:17:21.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.103 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2023-10-07T23:17:21.141 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:17:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:17:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:17:21.141 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2023-10-07T23:17:21.142 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:17:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2023-10-07T23:17:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-07T23:17:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41668sec preferred_lft 41668sec 2023-10-07T23:17:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2023-10-07T23:17:21.143 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:17:21.143 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-07T23:17:21.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.143 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:17:21.143 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:17:21.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2023-10-07T23:17:21.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2023-10-07T23:17:21.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2023-10-07T23:17:21.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-07T23:17:21.144 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:17:21.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:21 smithi067 ceph-mon[82723]: pgmap v347: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:21.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:21 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1841295695' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-07T23:17:21.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.375 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:17:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:21 smithi123 ceph-mon[122926]: pgmap v347: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:21.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:21 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1841295695' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-07T23:17:21.439 INFO:teuthology.orchestra.run.smithi067.stdout:1 2023-10-07T23:17:21.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.443 DEBUG:teuthology.orchestra.run.smithi067:> route 2023-10-07T23:17:21.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:21 smithi086 ceph-mon[122131]: pgmap v347: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-07T23:17:21.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:21 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1841295695' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-07T23:17:21.482 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2023-10-07T23:17:21.482 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:17:21.482 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T23:17:21.482 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-07T23:17:21.483 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:17:21.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.484 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:17:21.484 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:17:21.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T23:17:21.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T23:17:21.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T23:17:21.484 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:17:21.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.685 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-07T23:17:21.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.722 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2023-10-07T23:17:21.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.798 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:17:21.798 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:17:21.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:17:21.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-07T23:17:21.798 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:17:21.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-07T23:17:21.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-07T23:17:21.969 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:17:22.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:22.208 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-07T23:17:22.208 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-07T23:17:22.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2023-10-07T23:17:22.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2023-10-07T23:17:22.208 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-07T23:17:22.368 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:17:22.368 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T23:17:22.405 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:17:22.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:17:22.406 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:17:22.481 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-07T23:17:22.481 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:17:22.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-10-07T23:17:22.614 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-10-07T23:17:22.614 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2023-10-07T23:17:23.109 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-10-07T23:17:23.109 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2023-10-07T23:17:23.109 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.86:3300,172.21.15.123:3300" 2023-10-07T23:17:23.110 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@227de690-6566-11ee-8db5-212e2dc638e7.cephfs=/ 2023-10-07T23:17:23.110 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.67:3300/172.21.15.86:3300/172.21.15.123:3300" will pass to kernel 2023-10-07T23:17:23.114 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-07T23:17:23.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:23.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:17:23.178 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-07T23:17:23.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:17:23.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:23 smithi067 ceph-mon[82723]: pgmap v348: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 -- 172.21.15.67:0/2384552631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414071bc0 msgr2=0x7fb414070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/2384552631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414071bc0 0x7fb414070030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb410004900 tx=0x7fb410009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 -- 172.21.15.67:0/2384552631 shutdown_connections 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/2384552631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414104b90 0x7fb414107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/2384552631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414071bc0 0x7fb414070030 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/2384552631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb414070570 0x7fb4140709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 -- 172.21.15.67:0/2384552631 >> 172.21.15.67:0/2384552631 conn(0x7fb4140f8410 msgr2=0x7fb4140fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 -- 172.21.15.67:0/2384552631 shutdown_connections 2023-10-07T23:17:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.390+0000 7fb41ab9d700 1 -- 172.21.15.67:0/2384552631 wait complete. 2023-10-07T23:17:23.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:23 smithi123 ceph-mon[122926]: pgmap v348: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:23.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 Processor -- start 2023-10-07T23:17:23.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 -- start start 2023-10-07T23:17:23.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 0x7fb41410fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb414071bc0 0x7fb414110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414104b90 0x7fb414117270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4141178b0 con 0x7fb414070570 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414117a20 con 0x7fb414071bc0 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.391+0000 7fb41ab9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb414117be0 con 0x7fb414104b90 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 0x7fb41410fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb41a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414104b90 0x7fb414117270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 0x7fb41410fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45996/0 (socket says 172.21.15.67:45996) 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb41939a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb414071bc0 0x7fb414110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 -- 172.21.15.67:0/1130775841 learned_addr learned my addr 172.21.15.67:0/1130775841 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 -- 172.21.15.67:0/1130775841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414104b90 msgr2=0x7fb414117270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414104b90 0x7fb414117270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 -- 172.21.15.67:0/1130775841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb414071bc0 msgr2=0x7fb414110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb414071bc0 0x7fb414110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 -- 172.21.15.67:0/1130775841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410005040 con 0x7fb414070570 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.392+0000 7fb419b9b700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 0x7fb41410fce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb410009040 tx=0x7fb410009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.393+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e040 con 0x7fb414070570 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.393+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb410009580 con 0x7fb414070570 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.393+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb414117e60 con 0x7fb414070570 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.393+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100123c0 con 0x7fb414070570 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.393+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb414118200 con 0x7fb414070570 2023-10-07T23:17:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.394+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4141032e0 con 0x7fb414070570 2023-10-07T23:17:23.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.397+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fb41001c430 con 0x7fb414070570 2023-10-07T23:17:23.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.398+0000 7fb40a7fc700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb40009b7f0 0x7fb40009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.398+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fb4100c65f0 con 0x7fb414070570 2023-10-07T23:17:23.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.398+0000 7fb41939a700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb40009b7f0 0x7fb40009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.399+0000 7fb41939a700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb40009b7f0 0x7fb40009dca0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb404009b80 tx=0x7fb40400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.402+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fb410091b60 con 0x7fb414070570 2023-10-07T23:17:23.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:23 smithi086 ceph-mon[122131]: pgmap v348: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-07T23:17:23.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.606+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb41410a740 con 0x7fb414070570 2023-10-07T23:17:23.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.607+0000 7fb40a7fc700 1 -- 172.21.15.67:0/1130775841 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7fb410095180 con 0x7fb414070570 2023-10-07T23:17:23.608 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:23.608 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:17:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.609+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb40009b7f0 msgr2=0x7fb40009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb40009b7f0 0x7fb40009dca0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb404009b80 tx=0x7fb40400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 msgr2=0x7fb41410fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 0x7fb41410fce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb410009040 tx=0x7fb410009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 shutdown_connections 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fb40009b7f0 0x7fb40009dca0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414104b90 0x7fb414117270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb414070570 0x7fb41410fce0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 --2- 172.21.15.67:0/1130775841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb414071bc0 0x7fb414110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 >> 172.21.15.67:0/1130775841 conn(0x7fb4140f8410 msgr2=0x7fb4140fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 shutdown_connections 2023-10-07T23:17:23.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.610+0000 7fb41ab9d700 1 -- 172.21.15.67:0/1130775841 wait complete. 2023-10-07T23:17:23.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:15:25.227955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:15:26.323230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-07T23:15:29.861861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:17:23.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:17:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 -- 172.21.15.67:0/2058559141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fe890 msgr2=0x7fa734105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 --2- 172.21.15.67:0/2058559141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fe890 0x7fa734105150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0049c0 tx=0x7fa72c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 -- 172.21.15.67:0/2058559141 shutdown_connections 2023-10-07T23:17:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 --2- 172.21.15.67:0/2058559141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7340fccf0 0x7fa7340fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 --2- 172.21.15.67:0/2058559141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fe890 0x7fa734105150 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 --2- 172.21.15.67:0/2058559141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fdf00 0x7fa7340fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 -- 172.21.15.67:0/2058559141 >> 172.21.15.67:0/2058559141 conn(0x7fa7340f8450 msgr2=0x7fa7340fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 -- 172.21.15.67:0/2058559141 shutdown_connections 2023-10-07T23:17:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.820+0000 7fa73999d700 1 -- 172.21.15.67:0/2058559141 wait complete. 2023-10-07T23:17:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.821+0000 7fa73999d700 1 Processor -- start 2023-10-07T23:17:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.821+0000 7fa73999d700 1 -- start start 2023-10-07T23:17:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73999d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fccf0 0x7fa73410f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73999d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 0x7fa73410fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73999d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7340fe890 0x7fa734116f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73999d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa734117540 con 0x7fa7340fdf00 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73999d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341176b0 con 0x7fa7340fe890 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73999d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa734117870 con 0x7fa7340fccf0 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 0x7fa73410fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73899b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fccf0 0x7fa73410f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 0x7fa73410fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46010/0 (socket says 172.21.15.67:46010) 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73899b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fccf0 0x7fa73410f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:47698/0 (socket says 172.21.15.67:47698) 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 -- 172.21.15.67:0/3181765877 learned_addr learned my addr 172.21.15.67:0/3181765877 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa73919c700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7340fe890 0x7fa734116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 -- 172.21.15.67:0/3181765877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fccf0 msgr2=0x7fa73410f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fccf0 0x7fa73410f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 -- 172.21.15.67:0/3181765877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7340fe890 msgr2=0x7fa734116f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7340fe890 0x7fa734116f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.822+0000 7fa733fff700 1 -- 172.21.15.67:0/3181765877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280048f0 con 0x7fa7340fdf00 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.823+0000 7fa733fff700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 0x7fa73410fea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa72000ebf0 tx=0x7fa72000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.823+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200047d0 con 0x7fa7340fdf00 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.823+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72000f040 con 0x7fa7340fdf00 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.823+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c005040 con 0x7fa7340fdf00 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.823+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000c900 con 0x7fa7340fdf00 2023-10-07T23:17:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.823+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa734117ed0 con 0x7fa7340fdf00 2023-10-07T23:17:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.824+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa734108560 con 0x7fa7340fdf00 2023-10-07T23:17:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.826+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fa72000cd30 con 0x7fa7340fdf00 2023-10-07T23:17:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.827+0000 7fa7317fa700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa72409b7f0 0x7fa72409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.827+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fa7200c4d70 con 0x7fa7340fdf00 2023-10-07T23:17:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.827+0000 7fa73899b700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa72409b7f0 0x7fa72409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.828+0000 7fa73899b700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa72409b7f0 0x7fa72409dca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa7340fdd60 tx=0x7fa72800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:23.831+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fa720090220 con 0x7fa7340fdf00 2023-10-07T23:17:24.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.036+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa734118250 con 0x7fa7340fdf00 2023-10-07T23:17:24.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.036+0000 7fa7317fa700 1 -- 172.21.15.67:0/3181765877 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa720019020 con 0x7fa7340fdf00 2023-10-07T23:17:24.037 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa72409b7f0 msgr2=0x7fa72409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa72409b7f0 0x7fa72409dca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa7340fdd60 tx=0x7fa72800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 msgr2=0x7fa73410fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 0x7fa73410fea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa72000ebf0 tx=0x7fa72000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 shutdown_connections 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7340fe890 0x7fa734116f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fa72409b7f0 0x7fa72409dca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7340fccf0 0x7fa73410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 --2- 172.21.15.67:0/3181765877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7340fdf00 0x7fa73410fea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 >> 172.21.15.67:0/3181765877 conn(0x7fa7340f8450 msgr2=0x7fa7340fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 shutdown_connections 2023-10-07T23:17:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.039+0000 7fa73999d700 1 -- 172.21.15.67:0/3181765877 wait complete. 2023-10-07T23:17:24.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 -- 172.21.15.67:0/4210218966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fe890 msgr2=0x7f213c105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 --2- 172.21.15.67:0/4210218966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fe890 0x7f213c105150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d60 tx=0x7f2134004d10 comp rx=0 tx=0).stop 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 -- 172.21.15.67:0/4210218966 shutdown_connections 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 --2- 172.21.15.67:0/4210218966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fccf0 0x7f213c0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 --2- 172.21.15.67:0/4210218966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fe890 0x7f213c105150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 --2- 172.21.15.67:0/4210218966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f213c0fdf00 0x7f213c0fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 -- 172.21.15.67:0/4210218966 >> 172.21.15.67:0/4210218966 conn(0x7f213c0f8450 msgr2=0x7f213c0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 -- 172.21.15.67:0/4210218966 shutdown_connections 2023-10-07T23:17:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.250+0000 7f213bfff700 1 -- 172.21.15.67:0/4210218966 wait complete. 2023-10-07T23:17:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 Processor -- start 2023-10-07T23:17:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 -- start start 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 0x7f213c10f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fdf00 0x7f213c10ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f213c0fe890 0x7f213c116fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1175c0 con 0x7f213c0fccf0 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c117730 con 0x7f213c0fdf00 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1178f0 con 0x7f213c0fe890 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 0x7f213c10f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fdf00 0x7f213c10ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.251+0000 7f213affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 0x7f213c10f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46028/0 (socket says 172.21.15.67:46028) 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213affd700 1 -- 172.21.15.67:0/57622118 learned_addr learned my addr 172.21.15.67:0/57622118 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213b7fe700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f213c0fe890 0x7f213c116fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 -- 172.21.15.67:0/57622118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f213c0fe890 msgr2=0x7f213c116fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f213c0fe890 0x7f213c116fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 -- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 msgr2=0x7f213c10f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 0x7f213c10f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 -- 172.21.15.67:0/57622118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2134009940 con 0x7f213c0fdf00 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213affd700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 0x7f213c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213a7fc700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fdf00 0x7f213c10ff20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2130009fd0 tx=0x7f213000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130007cc0 con 0x7f213c0fdf00 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213000f040 con 0x7f213c0fdf00 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c117b70 con 0x7f213c0fdf00 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.252+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000c940 con 0x7f213c0fdf00 2023-10-07T23:17:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.253+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c118060 con 0x7f213c0fdf00 2023-10-07T23:17:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.253+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c108540 con 0x7f213c0fdf00 2023-10-07T23:17:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.256+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f21300090d0 con 0x7f213c0fdf00 2023-10-07T23:17:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.257+0000 7f21237fe700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f212409b770 0x7f212409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.257+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f21300c5ad0 con 0x7f213c0fdf00 2023-10-07T23:17:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.257+0000 7f213affd700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f212409b770 0x7f212409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.258+0000 7f213affd700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f212409b770 0x7f212409dc20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f212c00f6a0 tx=0x7f212c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.259+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2130090f80 con 0x7f213c0fdf00 2023-10-07T23:17:24.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:24 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1130775841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:24.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:24 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1130775841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.460+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f213c118490 con 0x7f213c0fdf00 2023-10-07T23:17:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.461+0000 7f21237fe700 1 -- 172.21.15.67:0/57622118 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21300945a0 con 0x7f213c0fdf00 2023-10-07T23:17:24.462 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f212409b770 msgr2=0x7f212409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f212409b770 0x7f212409dc20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f212c00f6a0 tx=0x7f212c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fdf00 msgr2=0x7f213c10ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fdf00 0x7f213c10ff20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2130009fd0 tx=0x7f213000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 shutdown_connections 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f212409b770 0x7f212409dc20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f213c0fe890 0x7f213c116fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f213c0fccf0 0x7f213c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 --2- 172.21.15.67:0/57622118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c0fdf00 0x7f213c10ff20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 >> 172.21.15.67:0/57622118 conn(0x7f213c0f8450 msgr2=0x7f213c0fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 shutdown_connections 2023-10-07T23:17:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.464+0000 7f213bfff700 1 -- 172.21.15.67:0/57622118 wait complete. 2023-10-07T23:17:24.480 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:24 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1130775841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:24.482 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:17:24.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 -- 172.21.15.67:0/64774136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fdf00 msgr2=0x7fede40fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 --2- 172.21.15.67:0/64774136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fdf00 0x7fede40fe350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fedd0004900 tx=0x7fedd0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 -- 172.21.15.67:0/64774136 shutdown_connections 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 --2- 172.21.15.67:0/64774136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fede40fe890 0x7fede4105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 --2- 172.21.15.67:0/64774136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede40fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 --2- 172.21.15.67:0/64774136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fdf00 0x7fede40fe350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.673+0000 7fede9d9d700 1 -- 172.21.15.67:0/64774136 >> 172.21.15.67:0/64774136 conn(0x7fede40f8450 msgr2=0x7fede40fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.674+0000 7fede9d9d700 1 -- 172.21.15.67:0/64774136 shutdown_connections 2023-10-07T23:17:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.674+0000 7fede9d9d700 1 -- 172.21.15.67:0/64774136 wait complete. 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.674+0000 7fede9d9d700 1 Processor -- start 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.674+0000 7fede9d9d700 1 -- start start 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede9d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede4071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede9d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fede40fdf00 0x7fede406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede9d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fe890 0x7fede40704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede9d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4070c40 con 0x7fede40fccf0 2023-10-07T23:17:24.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede9d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4070db0 con 0x7fede40fdf00 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede9d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4117a90 con 0x7fede40fe890 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede4071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46034/0 (socket says 172.21.15.67:46034) 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 -- 172.21.15.67:0/72453563 learned_addr learned my addr 172.21.15.67:0/72453563 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede3fff700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fede40fdf00 0x7fede406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede959c700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fe890 0x7fede40704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 -- 172.21.15.67:0/72453563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fe890 msgr2=0x7fede40704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fe890 0x7fede40704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 -- 172.21.15.67:0/72453563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fede40fdf00 msgr2=0x7fede406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fede40fdf00 0x7fede406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.675+0000 7fede8d9b700 1 -- 172.21.15.67:0/72453563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8005040 con 0x7fede40fccf0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.676+0000 7fede8d9b700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede4071930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009730 tx=0x7fedd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.676+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8007be0 con 0x7fede40fccf0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.676+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd800f040 con 0x7fede40fccf0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.676+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd0005040 con 0x7fede40fccf0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.676+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8013c30 con 0x7fede40fccf0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.676+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede4118090 con 0x7fede40fccf0 2023-10-07T23:17:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.677+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede4108560 con 0x7fede40fccf0 2023-10-07T23:17:24.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.679+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fedd800e840 con 0x7fede40fccf0 2023-10-07T23:17:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.680+0000 7fede17fa700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fedd409b7f0 0x7fedd409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.680+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fedd80c5100 con 0x7fede40fccf0 2023-10-07T23:17:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.681+0000 7fede3fff700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fedd409b7f0 0x7fedd409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:24.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.681+0000 7fede3fff700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fedd409b7f0 0x7fedd409dca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fedd0004f70 tx=0x7fedd0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:24.685 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.684+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fedd80905b0 con 0x7fede40fccf0 2023-10-07T23:17:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.889+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fede41184e0 con 0x7fede40fccf0 2023-10-07T23:17:24.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.890+0000 7fede17fa700 1 -- 172.21.15.67:0/72453563 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedd8093bd0 con 0x7fede40fccf0 2023-10-07T23:17:24.891 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fedd409b7f0 msgr2=0x7fedd409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fedd409b7f0 0x7fedd409dca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fedd0004f70 tx=0x7fedd0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 msgr2=0x7fede4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede4071930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009730 tx=0x7fedd8009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 shutdown_connections 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fedd409b7f0 0x7fedd409dca0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fede40fe890 0x7fede40704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fede40fccf0 0x7fede4071930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 --2- 172.21.15.67:0/72453563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fede40fdf00 0x7fede406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:24.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 >> 172.21.15.67:0/72453563 conn(0x7fede40f8450 msgr2=0x7fede40fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:24.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.892+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 shutdown_connections 2023-10-07T23:17:24.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:24.893+0000 7fede9d9d700 1 -- 172.21.15.67:0/72453563 wait complete. 2023-10-07T23:17:24.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-07T23:17:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 -- 172.21.15.67:0/843044194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 msgr2=0x7efe080fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/843044194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe080fd140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efe040049c0 tx=0x7efe04009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 -- 172.21.15.67:0/843044194 shutdown_connections 2023-10-07T23:17:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/843044194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080fe8d0 0x7efe08105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/843044194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe080fd140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/843044194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe080fdf40 0x7efe080fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 -- 172.21.15.67:0/843044194 >> 172.21.15.67:0/843044194 conn(0x7efe080f8490 msgr2=0x7efe080fa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:25.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 -- 172.21.15.67:0/843044194 shutdown_connections 2023-10-07T23:17:25.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.106+0000 7efe0ed9d700 1 -- 172.21.15.67:0/843044194 wait complete. 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.107+0000 7efe0ed9d700 1 Processor -- start 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.107+0000 7efe0ed9d700 1 -- start start 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.107+0000 7efe0ed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe08071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.107+0000 7efe0ed9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe080fdf40 0x7efe0806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.107+0000 7efe0ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080fe8d0 0x7efe080704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0ed9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08070c40 con 0x7efe080fcd30 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0ed9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08070db0 con 0x7efe080fdf40 2023-10-07T23:17:25.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0ed9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08117a80 con 0x7efe080fe8d0 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe08071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe08071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46054/0 (socket says 172.21.15.67:46054) 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0e59c700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080fe8d0 0x7efe080704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 -- 172.21.15.67:0/544489302 learned_addr learned my addr 172.21.15.67:0/544489302 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0d59a700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe080fdf40 0x7efe0806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 -- 172.21.15.67:0/544489302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080fe8d0 msgr2=0x7efe080704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080fe8d0 0x7efe080704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 -- 172.21.15.67:0/544489302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe080fdf40 msgr2=0x7efe0806ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe080fdf40 0x7efe0806ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 -- 172.21.15.67:0/544489302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8005040 con 0x7efe080fcd30 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.108+0000 7efe0dd9b700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe08071930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe04009b30 tx=0x7efe04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.109+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400e050 con 0x7efe080fcd30 2023-10-07T23:17:25.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.109+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0401b3f0 con 0x7efe080fcd30 2023-10-07T23:17:25.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.109+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04012630 con 0x7efe080fcd30 2023-10-07T23:17:25.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.109+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04005040 con 0x7efe080fcd30 2023-10-07T23:17:25.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.109+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe08117fd0 con 0x7efe080fcd30 2023-10-07T23:17:25.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.110+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe08108540 con 0x7efe080fcd30 2023-10-07T23:17:25.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.112+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7efe0401b560 con 0x7efe080fcd30 2023-10-07T23:17:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.113+0000 7efdfe7fc700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efdf409b7f0 0x7efdf409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.113+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7efe040c64c0 con 0x7efe080fcd30 2023-10-07T23:17:25.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.113+0000 7efe0d59a700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efdf409b7f0 0x7efdf409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.114+0000 7efe0d59a700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efdf409b7f0 0x7efdf409dca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009fd0 tx=0x7efdf800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:25.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.116+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7efe04091970 con 0x7efe080fcd30 2023-10-07T23:17:25.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:25 smithi067 ceph-mon[82723]: pgmap v349: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-10-07T23:17:25.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:25 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3181765877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:25 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/57622118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:25 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/72453563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.333+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efe081183e0 con 0x7efe080fcd30 2023-10-07T23:17:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.333+0000 7efdfe7fc700 1 -- 172.21.15.67:0/544489302 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7efe04094f90 con 0x7efe080fcd30 2023-10-07T23:17:25.334 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efdf409b7f0 msgr2=0x7efdf409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efdf409b7f0 0x7efdf409dca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009fd0 tx=0x7efdf800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 msgr2=0x7efe08071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe08071930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe04009b30 tx=0x7efe04009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 shutdown_connections 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efdf409b7f0 0x7efdf409dca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080fe8d0 0x7efe080704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe080fcd30 0x7efe08071930 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 --2- 172.21.15.67:0/544489302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe080fdf40 0x7efe0806ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 >> 172.21.15.67:0/544489302 conn(0x7efe080f8490 msgr2=0x7efe080fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:25.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 shutdown_connections 2023-10-07T23:17:25.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.336+0000 7efe0ed9d700 1 -- 172.21.15.67:0/544489302 wait complete. 2023-10-07T23:17:25.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:17:25.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:25 smithi123 ceph-mon[122926]: pgmap v349: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-10-07T23:17:25.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:25 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3181765877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:25 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/57622118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:25 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/72453563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:25 smithi086 ceph-mon[122131]: pgmap v349: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-10-07T23:17:25.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:25 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3181765877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:25 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/57622118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:25 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/72453563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 -- 172.21.15.67:0/2323986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 msgr2=0x7f65340709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 --2- 172.21.15.67:0/2323986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f65340709c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6528004900 tx=0x7f6528009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 -- 172.21.15.67:0/2323986 shutdown_connections 2023-10-07T23:17:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 --2- 172.21.15.67:0/2323986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6534071bc0 0x7f6534070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 --2- 172.21.15.67:0/2323986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534104b80 0x7f6534107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 --2- 172.21.15.67:0/2323986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f65340709c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 -- 172.21.15.67:0/2323986 >> 172.21.15.67:0/2323986 conn(0x7f65340f8460 msgr2=0x7f65340fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 -- 172.21.15.67:0/2323986 shutdown_connections 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.544+0000 7f653959d700 1 -- 172.21.15.67:0/2323986 wait complete. 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.545+0000 7f653959d700 1 Processor -- start 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.545+0000 7f653959d700 1 -- start start 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.545+0000 7f653959d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f653410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.545+0000 7f653959d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534071bc0 0x7f6534110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f653959d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6534104b80 0x7f6534117110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f653959d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6534117720 con 0x7f6534071bc0 2023-10-07T23:17:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f653959d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6534117890 con 0x7f6534104b80 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f653959d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6534117a50 con 0x7f6534070570 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f65337fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534071bc0 0x7f6534110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f653410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f653410fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:47792/0 (socket says 172.21.15.67:47792) 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f65337fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534071bc0 0x7f6534110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46080/0 (socket says 172.21.15.67:46080) 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 -- 172.21.15.67:0/1449616913 learned_addr learned my addr 172.21.15.67:0/1449616913 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6538d9c700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6534104b80 0x7f6534117110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 -- 172.21.15.67:0/1449616913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6534104b80 msgr2=0x7f6534117110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6534104b80 0x7f6534117110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 -- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534071bc0 msgr2=0x7f6534110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534071bc0 0x7f6534110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 -- 172.21.15.67:0/1449616913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528005040 con 0x7f6534070570 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.546+0000 7f6533fff700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f653410fb40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f652400ebf0 tx=0x7f652400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.547+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240047d0 con 0x7f6534070570 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.547+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f652400f040 con 0x7f6534070570 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.547+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6534117cd0 con 0x7f6534070570 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.547+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652400c980 con 0x7f6534070570 2023-10-07T23:17:25.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.547+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6534118170 con 0x7f6534070570 2023-10-07T23:17:25.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.548+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65341032d0 con 0x7f6534070570 2023-10-07T23:17:25.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.550+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f65240090d0 con 0x7f6534070570 2023-10-07T23:17:25.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.551+0000 7f6530ff9700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f651c09b7f0 0x7f651c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.551+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f65240c50d0 con 0x7f6534070570 2023-10-07T23:17:25.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.551+0000 7f65337fe700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f651c09b7f0 0x7f651c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.552+0000 7f65337fe700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f651c09b7f0 0x7f651c09dca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6528004f70 tx=0x7f6528009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.553+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f6524090580 con 0x7f6534070570 2023-10-07T23:17:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.761+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f653410a730 con 0x7f6534070570 2023-10-07T23:17:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.762+0000 7f6530ff9700 1 -- 172.21.15.67:0/1449616913 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+23637 (secure 0 0 0) 0x7f6524019070 con 0x7f6534070570 2023-10-07T23:17:25.765 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:25.766 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:16:28.996447+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:17:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.764+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f651c09b7f0 msgr2=0x7f651c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.764+0000 7f653959d700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f651c09b7f0 0x7f651c09dca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6528004f70 tx=0x7f6528009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.764+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 msgr2=0x7f653410fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.764+0000 7f653959d700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f653410fb40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f652400ebf0 tx=0x7f652400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.764+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 shutdown_connections 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6534104b80 0x7f6534117110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f651c09b7f0 0x7f651c09dca0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6534070570 0x7f653410fb40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 --2- 172.21.15.67:0/1449616913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6534071bc0 0x7f6534110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 >> 172.21.15.67:0/1449616913 conn(0x7f65340f8460 msgr2=0x7f6534106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 shutdown_connections 2023-10-07T23:17:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.765+0000 7f653959d700 1 -- 172.21.15.67:0/1449616913 wait complete. 2023-10-07T23:17:25.783 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:10:50.442097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:15:25.227955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:15:26.323230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-07T23:15:29.861861+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:17:25.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:17:25.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 -- 172.21.15.67:0/85757817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fe890 msgr2=0x7f2bac105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 --2- 172.21.15.67:0/85757817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fe890 0x7f2bac105150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009d60 tx=0x7f2ba4004d10 comp rx=0 tx=0).stop 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 -- 172.21.15.67:0/85757817 shutdown_connections 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 --2- 172.21.15.67:0/85757817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fe890 0x7f2bac105150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 --2- 172.21.15.67:0/85757817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bac0fccf0 0x7f2bac0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 --2- 172.21.15.67:0/85757817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac0fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 -- 172.21.15.67:0/85757817 >> 172.21.15.67:0/85757817 conn(0x7f2bac0f8410 msgr2=0x7f2bac0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 -- 172.21.15.67:0/85757817 shutdown_connections 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.976+0000 7f2babfff700 1 -- 172.21.15.67:0/85757817 wait complete. 2023-10-07T23:17:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.977+0000 7f2babfff700 1 Processor -- start 2023-10-07T23:17:25.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.977+0000 7f2babfff700 1 -- start start 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2babfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 0x7f2bac10fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2babfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac10ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2babfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bac0fe890 0x7f2bac116fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2babfff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac1175f0 con 0x7f2bac0fdf00 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2babfff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac117760 con 0x7f2bac0fccf0 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2babfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac117920 con 0x7f2bac0fe890 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baa7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac10ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 0x7f2bac10fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 0x7f2bac10fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:40340/0 (socket says 172.21.15.67:40340) 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baa7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac10ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46096/0 (socket says 172.21.15.67:46096) 2023-10-07T23:17:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 -- 172.21.15.67:0/3030949340 learned_addr learned my addr 172.21.15.67:0/3030949340 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2bab7fe700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bac0fe890 0x7f2bac116fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 -- 172.21.15.67:0/3030949340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bac0fe890 msgr2=0x7f2bac116fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bac0fe890 0x7f2bac116fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 -- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 msgr2=0x7f2bac10ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac10ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baaffd700 1 -- 172.21.15.67:0/3030949340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba0005040 con 0x7f2bac0fccf0 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.978+0000 7f2baa7fc700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.979+0000 7f2baaffd700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 0x7f2bac10fa10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00ebc0 tx=0x7f2b9c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.979+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0047d0 con 0x7f2bac0fccf0 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.979+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9c00f040 con 0x7f2bac0fccf0 2023-10-07T23:17:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.979+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4009940 con 0x7f2bac0fccf0 2023-10-07T23:17:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.979+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00c930 con 0x7f2bac0fccf0 2023-10-07T23:17:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.979+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac117f30 con 0x7f2bac0fccf0 2023-10-07T23:17:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.980+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac108560 con 0x7f2bac0fccf0 2023-10-07T23:17:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.982+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f2b9c041b00 con 0x7f2bac0fccf0 2023-10-07T23:17:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.983+0000 7f2b937fe700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b9409b770 0x7f2b9409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.984+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f2b9c0c4f50 con 0x7f2bac0fccf0 2023-10-07T23:17:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.984+0000 7f2baa7fc700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b9409b770 0x7f2b9409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.984+0000 7f2baa7fc700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b9409b770 0x7f2b9409dc20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009fd0 tx=0x7f2ba000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:25.986+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f2b9c090400 con 0x7f2bac0fccf0 2023-10-07T23:17:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.189+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bac118120 con 0x7f2bac0fccf0 2023-10-07T23:17:26.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.190+0000 7f2b937fe700 1 -- 172.21.15.67:0/3030949340 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b9c019070 con 0x7f2bac0fccf0 2023-10-07T23:17:26.190 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b9409b770 msgr2=0x7f2b9409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b9409b770 0x7f2b9409dc20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009fd0 tx=0x7f2ba000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 msgr2=0x7f2bac10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 0x7f2bac10fa10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00ebc0 tx=0x7f2b9c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 shutdown_connections 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bac0fccf0 0x7f2bac10fa10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f2b9409b770 0x7f2b9409dc20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bac0fe890 0x7f2bac116fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 --2- 172.21.15.67:0/3030949340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0fdf00 0x7f2bac10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 >> 172.21.15.67:0/3030949340 conn(0x7f2bac0f8410 msgr2=0x7f2bac0fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 shutdown_connections 2023-10-07T23:17:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.192+0000 7f2babfff700 1 -- 172.21.15.67:0/3030949340 wait complete. 2023-10-07T23:17:26.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:17:26.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/544489302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:26.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:26 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1449616913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 -- 172.21.15.67:0/1938732304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 msgr2=0x7f89bc105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 --2- 172.21.15.67:0/1938732304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 0x7f89bc105150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f89b4004ca0 tx=0x7f89b4009ca0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 -- 172.21.15.67:0/1938732304 shutdown_connections 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 --2- 172.21.15.67:0/1938732304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 0x7f89bc105150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 --2- 172.21.15.67:0/1938732304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 --2- 172.21.15.67:0/1938732304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 0x7f89bc0fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 -- 172.21.15.67:0/1938732304 >> 172.21.15.67:0/1938732304 conn(0x7f89bc0f8450 msgr2=0x7f89bc0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 -- 172.21.15.67:0/1938732304 shutdown_connections 2023-10-07T23:17:26.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.404+0000 7f89c259d700 1 -- 172.21.15.67:0/1938732304 wait complete. 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.405+0000 7f89c259d700 1 Processor -- start 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.405+0000 7f89c259d700 1 -- start start 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.405+0000 7f89c259d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc10f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.405+0000 7f89c259d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 0x7f89bc10fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c259d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 0x7f89bc116f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c259d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc117580 con 0x7f89bc0fdf00 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c259d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1176f0 con 0x7f89bc0fe890 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c259d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1178b0 con 0x7f89bc0fccf0 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc10f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c0d9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 0x7f89bc10fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc10f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:47840/0 (socket says 172.21.15.67:47840) 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 -- 172.21.15.67:0/4197126166 learned_addr learned my addr 172.21.15.67:0/4197126166 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c1d9c700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 0x7f89bc116f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 -- 172.21.15.67:0/4197126166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 msgr2=0x7f89bc116f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 0x7f89bc116f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 -- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 msgr2=0x7f89bc10fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 0x7f89bc10fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c159b700 1 -- 172.21.15.67:0/4197126166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b8005040 con 0x7f89bc0fccf0 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.406+0000 7f89c0d9a700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 0x7f89bc10fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.407+0000 7f89c159b700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc10f9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009730 tx=0x7f89b8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.407+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8007be0 con 0x7f89bc0fccf0 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.407+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b800f040 con 0x7f89bc0fccf0 2023-10-07T23:17:26.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.407+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac005040 con 0x7f89bc0fccf0 2023-10-07T23:17:26.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.407+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8013b80 con 0x7f89bc0fccf0 2023-10-07T23:17:26.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.407+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b40048f0 con 0x7f89bc0fccf0 2023-10-07T23:17:26.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.408+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89bc108540 con 0x7f89bc0fccf0 2023-10-07T23:17:26.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.410+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f89b800e840 con 0x7f89bc0fccf0 2023-10-07T23:17:26.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.411+0000 7f89b1ffb700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f89a809b7f0 0x7f89a809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.411+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f89b80c5570 con 0x7f89bc0fccf0 2023-10-07T23:17:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.411+0000 7f89c0d9a700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f89a809b7f0 0x7f89a809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.412+0000 7f89c0d9a700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f89a809b7f0 0x7f89a809dca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009fd0 tx=0x7f89ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:26.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.413+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f89b8090a20 con 0x7f89bc0fccf0 2023-10-07T23:17:26.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/544489302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:26.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:26 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1449616913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/544489302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:26.504 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:26 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1449616913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:26.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.617+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89bc117ff0 con 0x7f89bc0fccf0 2023-10-07T23:17:26.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.618+0000 7f89b1ffb700 1 -- 172.21.15.67:0/4197126166 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89b8094040 con 0x7f89bc0fccf0 2023-10-07T23:17:26.618 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:26.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f89a809b7f0 msgr2=0x7f89a809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f89a809b7f0 0x7f89a809dca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009fd0 tx=0x7f89ac00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 msgr2=0x7f89bc10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc10f9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009730 tx=0x7f89b8009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 shutdown_connections 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89bc0fe890 0x7f89bc116f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f89a809b7f0 0x7f89a809dca0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89bc0fccf0 0x7f89bc10f9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 --2- 172.21.15.67:0/4197126166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89bc0fdf00 0x7f89bc10fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 >> 172.21.15.67:0/4197126166 conn(0x7f89bc0f8450 msgr2=0x7f89bc0fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 shutdown_connections 2023-10-07T23:17:26.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.620+0000 7f89c259d700 1 -- 172.21.15.67:0/4197126166 wait complete. 2023-10-07T23:17:26.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:17:26.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 -- 172.21.15.67:0/3588553446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fdf00 msgr2=0x7f789c0fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 --2- 172.21.15.67:0/3588553446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fdf00 0x7f789c0fe350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f78900049c0 tx=0x7f7890009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:26.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 -- 172.21.15.67:0/3588553446 shutdown_connections 2023-10-07T23:17:26.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 --2- 172.21.15.67:0/3588553446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fe890 0x7f789c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 --2- 172.21.15.67:0/3588553446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0fccf0 0x7f789c0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 --2- 172.21.15.67:0/3588553446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fdf00 0x7f789c0fe350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 -- 172.21.15.67:0/3588553446 >> 172.21.15.67:0/3588553446 conn(0x7f789c0f8410 msgr2=0x7f789c0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:26.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 -- 172.21.15.67:0/3588553446 shutdown_connections 2023-10-07T23:17:26.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.831+0000 7f789b59e700 1 -- 172.21.15.67:0/3588553446 wait complete. 2023-10-07T23:17:26.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.832+0000 7f789b59e700 1 Processor -- start 2023-10-07T23:17:26.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.832+0000 7f789b59e700 1 -- start start 2023-10-07T23:17:26.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.832+0000 7f789b59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 0x7f789c10d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.832+0000 7f789b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fdf00 0x7f789c10df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0fe890 0x7f789c114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789b59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c1155d0 con 0x7f789c0fccf0 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789b59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c115740 con 0x7f789c0fdf00 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c115900 con 0x7f789c0fe890 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 0x7f789c10d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 0x7f789c10d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46132/0 (socket says 172.21.15.67:46132) 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789a59c700 1 -- 172.21.15.67:0/285697511 learned_addr learned my addr 172.21.15.67:0/285697511 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fdf00 0x7f789c10df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789ad9d700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0fe890 0x7f789c114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 -- 172.21.15.67:0/285697511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0fe890 msgr2=0x7f789c114fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0fe890 0x7f789c114fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 -- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 msgr2=0x7f789c10d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 0x7f789c10d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 -- 172.21.15.67:0/285697511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890005040 con 0x7f789c0fdf00 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f789a59c700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 0x7f789c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.833+0000 7f7899d9b700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fdf00 0x7f789c10df30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f78900049c0 tx=0x7f7890009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.834+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000e050 con 0x7f789c0fdf00 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.834+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c115b80 con 0x7f789c0fdf00 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.834+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789001b3f0 con 0x7f789c0fdf00 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.834+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890012630 con 0x7f789c0fdf00 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.834+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c115fe0 con 0x7f789c0fdf00 2023-10-07T23:17:26.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.835+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c103b60 con 0x7f789c0fdf00 2023-10-07T23:17:26.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.837+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f7890012a60 con 0x7f789c0fdf00 2023-10-07T23:17:26.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.838+0000 7f788affd700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f788409b770 0x7f788409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:26.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.839+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f78900c6930 con 0x7f789c0fdf00 2023-10-07T23:17:26.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.839+0000 7f789a59c700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f788409b770 0x7f788409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:26.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.839+0000 7f789a59c700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f788409b770 0x7f788409dc20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f788c004fd0 tx=0x7f788c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:26.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:26.840+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7890091de0 con 0x7f789c0fdf00 2023-10-07T23:17:27.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.044+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f789c02b200 con 0x7f789c0fdf00 2023-10-07T23:17:27.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.050+0000 7f788affd700 1 -- 172.21.15.67:0/285697511 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7890028070 con 0x7f789c0fdf00 2023-10-07T23:17:27.051 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-07T23:17:27.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f788409b770 msgr2=0x7f788409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f788409b770 0x7f788409dc20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f788c004fd0 tx=0x7f788c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fdf00 msgr2=0x7f789c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fdf00 0x7f789c10df30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f78900049c0 tx=0x7f7890009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 shutdown_connections 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f788409b770 0x7f788409dc20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789c0fe890 0x7f789c114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f789c0fccf0 0x7f789c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 --2- 172.21.15.67:0/285697511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f789c0fdf00 0x7f789c10df30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 >> 172.21.15.67:0/285697511 conn(0x7f789c0f8410 msgr2=0x7f789c0fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 shutdown_connections 2023-10-07T23:17:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.053+0000 7f789b59e700 1 -- 172.21.15.67:0/285697511 wait complete. 2023-10-07T23:17:27.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-07T23:17:27.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.267+0000 7feffb39d700 1 -- 172.21.15.67:0/2599124698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fccf0 msgr2=0x7feff40fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.267+0000 7feffb39d700 1 --2- 172.21.15.67:0/2599124698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fccf0 0x7feff40fd100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feff00049c0 tx=0x7feff0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.267+0000 7feffb39d700 1 -- 172.21.15.67:0/2599124698 shutdown_connections 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.267+0000 7feffb39d700 1 --2- 172.21.15.67:0/2599124698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feff40fe890 0x7feff4105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.267+0000 7feffb39d700 1 --2- 172.21.15.67:0/2599124698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fccf0 0x7feff40fd100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.267+0000 7feffb39d700 1 --2- 172.21.15.67:0/2599124698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff40fdf00 0x7feff40fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.268+0000 7feffb39d700 1 -- 172.21.15.67:0/2599124698 >> 172.21.15.67:0/2599124698 conn(0x7feff40f8450 msgr2=0x7feff40fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.268+0000 7feffb39d700 1 -- 172.21.15.67:0/2599124698 shutdown_connections 2023-10-07T23:17:27.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.268+0000 7feffb39d700 1 -- 172.21.15.67:0/2599124698 wait complete. 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.268+0000 7feffb39d700 1 Processor -- start 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.268+0000 7feffb39d700 1 -- start start 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffb39d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feff40fccf0 0x7feff410fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffb39d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff40fdf00 0x7feff4110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffb39d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 0x7feff41171e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffb39d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4117820 con 0x7feff40fe890 2023-10-07T23:17:27.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffb39d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4117990 con 0x7feff40fdf00 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffb39d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4117b50 con 0x7feff40fccf0 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 0x7feff41171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 0x7feff41171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46150/0 (socket says 172.21.15.67:46150) 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 -- 172.21.15.67:0/1171881599 learned_addr learned my addr 172.21.15.67:0/1171881599 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffa39b700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feff40fccf0 0x7feff410fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feff9b9a700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff40fdf00 0x7feff4110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 -- 172.21.15.67:0/1171881599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feff40fccf0 msgr2=0x7feff410fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feff40fccf0 0x7feff410fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 -- 172.21.15.67:0/1171881599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff40fdf00 msgr2=0x7feff4110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff40fdf00 0x7feff4110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.269+0000 7feffab9c700 1 -- 172.21.15.67:0/1171881599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0048f0 con 0x7feff40fe890 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.270+0000 7feffab9c700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 0x7feff41171e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fefec007f00 tx=0x7fefec009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.270+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec004c50 con 0x7feff40fe890 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.270+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec00f040 con 0x7feff40fe890 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.270+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec012360 con 0x7feff40fe890 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.270+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff0005040 con 0x7feff40fe890 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.270+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff4118140 con 0x7feff40fe890 2023-10-07T23:17:27.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.271+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff410a550 con 0x7feff40fe890 2023-10-07T23:17:27.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.273+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fefec013600 con 0x7feff40fe890 2023-10-07T23:17:27.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.274+0000 7fefeaffd700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fefe009b7f0 0x7fefe009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.274+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7fefec0c5770 con 0x7feff40fe890 2023-10-07T23:17:27.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.274+0000 7feffa39b700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fefe009b7f0 0x7fefe009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.275+0000 7feffa39b700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fefe009b7f0 0x7fefe009dca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feff0004890 tx=0x7feff0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:27.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.278+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fefec090c20 con 0x7feff40fe890 2023-10-07T23:17:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:27 smithi123 ceph-mon[122926]: pgmap v350: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-07T23:17:27.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:27 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3030949340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:27.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:27 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/4197126166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:27 smithi086 ceph-mon[122131]: pgmap v350: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-07T23:17:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3030949340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:27.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:27 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/4197126166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.481+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feff402b420 con 0x7feff40fe890 2023-10-07T23:17:27.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.481+0000 7fefeaffd700 1 -- 172.21.15.67:0/1171881599 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fefec094240 con 0x7feff40fe890 2023-10-07T23:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-07T23:17:27.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fefe009b7f0 msgr2=0x7fefe009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fefe009b7f0 0x7fefe009dca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feff0004890 tx=0x7feff0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:27.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 msgr2=0x7feff41171e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 0x7feff41171e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fefec007f00 tx=0x7fefec009ca0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 shutdown_connections 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fefe009b7f0 0x7fefe009dca0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff40fe890 0x7feff41171e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feff40fccf0 0x7feff410fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 --2- 172.21.15.67:0/1171881599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feff40fdf00 0x7feff4110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 >> 172.21.15.67:0/1171881599 conn(0x7feff40f8450 msgr2=0x7feff4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 shutdown_connections 2023-10-07T23:17:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.484+0000 7feffb39d700 1 -- 172.21.15.67:0/1171881599 wait complete. 2023-10-07T23:17:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:27 smithi067 ceph-mon[82723]: pgmap v350: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-07T23:17:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3030949340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-07T23:17:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:27 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/4197126166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-07T23:17:27.502 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-07T23:17:27.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:17:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 -- 172.21.15.67:0/2821125037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fdef0 msgr2=0x7f22740fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 --2- 172.21.15.67:0/2821125037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fdef0 0x7f22740fe340 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22640049c0 tx=0x7f2264009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 -- 172.21.15.67:0/2821125037 shutdown_connections 2023-10-07T23:17:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 --2- 172.21.15.67:0/2821125037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fcce0 0x7f22740fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 --2- 172.21.15.67:0/2821125037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22740fe880 0x7f2274105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 --2- 172.21.15.67:0/2821125037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fdef0 0x7f22740fe340 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 -- 172.21.15.67:0/2821125037 >> 172.21.15.67:0/2821125037 conn(0x7f22740f8460 msgr2=0x7f22740fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:27.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 -- 172.21.15.67:0/2821125037 shutdown_connections 2023-10-07T23:17:27.692 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.690+0000 7f227b59d700 1 -- 172.21.15.67:0/2821125037 wait complete. 2023-10-07T23:17:27.694 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.691+0000 7f227b59d700 1 Processor -- start 2023-10-07T23:17:27.694 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.691+0000 7f227b59d700 1 -- start start 2023-10-07T23:17:27.694 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227b59d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22740fcce0 0x7f227410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227b59d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fdef0 0x7f227410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227b59d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 0x7f2274114fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227b59d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741155b0 con 0x7f22740fe880 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227b59d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274115720 con 0x7f22740fdef0 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227b59d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741158e0 con 0x7f22740fcce0 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 0x7f2274114fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 0x7f2274114fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46160/0 (socket says 172.21.15.67:46160) 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 -- 172.21.15.67:0/2578655862 learned_addr learned my addr 172.21.15.67:0/2578655862 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227a59b700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22740fcce0 0x7f227410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f2279d9a700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fdef0 0x7f227410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 -- 172.21.15.67:0/2578655862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22740fcce0 msgr2=0x7f227410d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22740fcce0 0x7f227410d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 -- 172.21.15.67:0/2578655862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fdef0 msgr2=0x7f227410df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fdef0 0x7f227410df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 -- 172.21.15.67:0/2578655862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264005040 con 0x7f22740fe880 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f2279d9a700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fdef0 0x7f227410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.692+0000 7f227ad9c700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 0x7f2274114fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f226c00ec30 tx=0x7f226c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.693+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c007b30 con 0x7f22740fe880 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.693+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2274115bc0 con 0x7f22740fe880 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.693+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f226c00f040 con 0x7f22740fe880 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.693+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00ca90 con 0x7f22740fe880 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.693+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2274116100 con 0x7f22740fe880 2023-10-07T23:17:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.694+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2274103bf0 con 0x7f22740fe880 2023-10-07T23:17:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.696+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f226c01d400 con 0x7f22740fe880 2023-10-07T23:17:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.697+0000 7f226affd700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f226009b770 0x7f226009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.697+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9016+0+0 (secure 0 0 0) 0x7f226c0c5770 con 0x7f22740fe880 2023-10-07T23:17:27.698 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.697+0000 7f227a59b700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f226009b770 0x7f226009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:27.698 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.698+0000 7f227a59b700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f226009b770 0x7f226009dc20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2270009b80 tx=0x7f227000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:27.701 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.701+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f226c090c20 con 0x7f22740fe880 2023-10-07T23:17:27.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.954+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f227402b3e0 con 0x7f22740fe880 2023-10-07T23:17:27.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.954+0000 7f226affd700 1 -- 172.21.15.67:0/2578655862 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f226c094240 con 0x7f22740fe880 2023-10-07T23:17:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:27.955 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:17:27.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f226009b770 msgr2=0x7f226009dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f226009b770 0x7f226009dc20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2270009b80 tx=0x7f227000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 msgr2=0x7f2274114fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 0x7f2274114fa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f226c00ec30 tx=0x7f226c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 shutdown_connections 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f226009b770 0x7f226009dc20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22740fe880 0x7f2274114fa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22740fcce0 0x7f227410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 --2- 172.21.15.67:0/2578655862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22740fdef0 0x7f227410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 >> 172.21.15.67:0/2578655862 conn(0x7f22740f8460 msgr2=0x7f22740fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:27.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 shutdown_connections 2023-10-07T23:17:27.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:27.957+0000 7f227b59d700 1 -- 172.21.15.67:0/2578655862 wait complete. 2023-10-07T23:17:27.959 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:17:27.976 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-07T23:17:27.976 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-07T23:17:27.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 -- 172.21.15.67:0/2074672672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c0ffe70 msgr2=0x7f441c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 --2- 172.21.15.67:0/2074672672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c0ffe70 0x7f441c1022d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f440c0049c0 tx=0x7f440c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 -- 172.21.15.67:0/2074672672 shutdown_connections 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 --2- 172.21.15.67:0/2074672672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f441c065ec0 0x7f441c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 --2- 172.21.15.67:0/2074672672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c102960 0x7f441c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 --2- 172.21.15.67:0/2074672672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c0ffe70 0x7f441c1022d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 -- 172.21.15.67:0/2074672672 >> 172.21.15.67:0/2074672672 conn(0x7f441c0f7310 msgr2=0x7f441c0f9780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:28.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 -- 172.21.15.67:0/2074672672 shutdown_connections 2023-10-07T23:17:28.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.169+0000 7f442299d700 1 -- 172.21.15.67:0/2074672672 wait complete. 2023-10-07T23:17:28.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.170+0000 7f442299d700 1 Processor -- start 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.170+0000 7f442299d700 1 -- start start 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.170+0000 7f442299d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 0x7f441c10fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442299d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c0ffe70 0x7f441c1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442299d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f441c102960 0x7f441c117170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442299d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c117780 con 0x7f441c0ffe70 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442299d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1178f0 con 0x7f441c102960 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442299d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c117ab0 con 0x7f441c065ec0 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c0ffe70 0x7f441c1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442199b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 0x7f441c10fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442199b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 0x7f441c10fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:47900/0 (socket says 172.21.15.67:47900) 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442199b700 1 -- 172.21.15.67:0/4222592449 learned_addr learned my addr 172.21.15.67:0/4222592449 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442219c700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f441c102960 0x7f441c117170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 -- 172.21.15.67:0/4222592449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 msgr2=0x7f441c10fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 0x7f441c10fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 -- 172.21.15.67:0/4222592449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f441c102960 msgr2=0x7f441c117170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f441c102960 0x7f441c117170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 -- 172.21.15.67:0/4222592449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c005040 con 0x7f441c0ffe70 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442199b700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 0x7f441c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.171+0000 7f442119a700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c0ffe70 0x7f441c1100e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f441c065c00 tx=0x7f440c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.172+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c00e050 con 0x7f441c0ffe70 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.172+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f440c01b3f0 con 0x7f441c0ffe70 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.172+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c117d30 con 0x7f441c0ffe70 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.172+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c012630 con 0x7f441c0ffe70 2023-10-07T23:17:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.172+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c118210 con 0x7f441c0ffe70 2023-10-07T23:17:28.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.173+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c108550 con 0x7f441c0ffe70 2023-10-07T23:17:28.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.175+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f440c01b560 con 0x7f441c0ffe70 2023-10-07T23:17:28.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.176+0000 7f44127fc700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f440809b7f0 0x7f440809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.176+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9103+0+0 (secure 0 0 0) 0x7f440c0c6070 con 0x7f441c0ffe70 2023-10-07T23:17:28.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.176+0000 7f442199b700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f440809b7f0 0x7f440809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.177+0000 7f442199b700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f440809b7f0 0x7f440809dca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4418007fd0 tx=0x7f441800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:28.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.179+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f440c0914d0 con 0x7f441c0ffe70 2023-10-07T23:17:28.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:28 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/285697511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:28.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:28 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1171881599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:28.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:28 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2578655862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:28.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-10-07T23:17:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-07T23:17:28.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:28 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-10-07T23:17:28.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.381+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f441c1046a0 con 0x7f440809b7f0 2023-10-07T23:17:28.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/285697511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:28.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1171881599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:28.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:28 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2578655862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:28.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-10-07T23:17:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-07T23:17:28.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:28 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-10-07T23:17:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.395+0000 7f44127fc700 1 -- 172.21.15.67:0/4222592449 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426905 (secure 0 0 0) 0x7f441c1046a0 con 0x7f440809b7f0 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f440809b7f0 msgr2=0x7f440809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f440809b7f0 0x7f440809dca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4418007fd0 tx=0x7f441800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c0ffe70 msgr2=0x7f441c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c0ffe70 0x7f441c1100e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f441c065c00 tx=0x7f440c009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 shutdown_connections 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f441c102960 0x7f441c117170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f440809b7f0 0x7f440809dca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f441c065ec0 0x7f441c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 --2- 172.21.15.67:0/4222592449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441c0ffe70 0x7f441c1100e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 >> 172.21.15.67:0/4222592449 conn(0x7f441c0f7310 msgr2=0x7f441c0f96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 shutdown_connections 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.400+0000 7f442299d700 1 -- 172.21.15.67:0/4222592449 wait complete. 2023-10-07T23:17:28.405 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:17:28.419 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2023-10-07T23:17:28.052053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600431,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1080,"num_read_kb":602,"num_write":392,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3581412,"kb_used_data":13956,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121360412,"statfs":{"total":1151940427776,"available":1148273061888,"internally_reserved":0,"allocated":14290944,"data_stored":7730295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":51,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.101980"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004139+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:16:28.004139+0000","last_peered":"2023-10-07T23:16:28.004139+0000","last_clean":"2023-10-07T23:16:28.004139+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:16:28.004139+0000","last_undegraded":"2023-10-07T23:16:28.004139+0000","last_fullsized":"2023-10-07T23:16:28.004139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012736+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:16:28.012736+0000","last_peered":"2023-10-07T23:16:28.012736+0000","last_clean":"2023-10-07T23:16:28.012736+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:16:28.012736+0000","last_undegraded":"2023-10-07T23:16:28.012736+0000","last_fullsized":"2023-10-07T23:16:28.012736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012507+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:16:28.012507+0000","last_peered":"2023-10-07T23:16:28.012507+0000","last_clean":"2023-10-07T23:16:28.012507+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:16:28.012507+0000","last_undegraded":"2023-10-07T23:16:28.012507+0000","last_fullsized":"2023-10-07T23:16:28.012507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999764+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:16:27.999764+0000","last_peered":"2023-10-07T23:16:27.999764+0000","last_clean":"2023-10-07T23:16:27.999764+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:16:27.999764+0000","last_undegraded":"2023-10-07T23:16:27.999764+0000","last_fullsized":"2023-10-07T23:16:27.999764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011235+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:16:28.011235+0000","last_peered":"2023-10-07T23:16:28.011235+0000","last_clean":"2023-10-07T23:16:28.011235+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:16:28.011235+0000","last_undegraded":"2023-10-07T23:16:28.011235+0000","last_fullsized":"2023-10-07T23:16:28.011235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003946+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:16:28.003946+0000","last_peered":"2023-10-07T23:16:28.003946+0000","last_clean":"2023-10-07T23:16:28.003946+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:16:28.003946+0000","last_undegraded":"2023-10-07T23:16:28.003946+0000","last_fullsized":"2023-10-07T23:16:28.003946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007260+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:16:28.007260+0000","last_peered":"2023-10-07T23:16:28.007260+0000","last_clean":"2023-10-07T23:16:28.007260+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:16:28.007260+0000","last_undegraded":"2023-10-07T23:16:28.007260+0000","last_fullsized":"2023-10-07T23:16:28.007260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015359+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:16:28.015359+0000","last_peered":"2023-10-07T23:16:28.015359+0000","last_clean":"2023-10-07T23:16:28.015359+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:16:28.015359+0000","last_undegraded":"2023-10-07T23:16:28.015359+0000","last_fullsized":"2023-10-07T23:16:28.015359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006337+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:16:28.006337+0000","last_peered":"2023-10-07T23:16:28.006337+0000","last_clean":"2023-10-07T23:16:28.006337+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:16:28.006337+0000","last_undegraded":"2023-10-07T23:16:28.006337+0000","last_fullsized":"2023-10-07T23:16:28.006337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003351+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:16:28.003351+0000","last_peered":"2023-10-07T23:16:28.003351+0000","last_clean":"2023-10-07T23:16:28.003351+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:16:28.003351+0000","last_undegraded":"2023-10-07T23:16:28.003351+0000","last_fullsized":"2023-10-07T23:16:28.003351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007883+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:16:28.007883+0000","last_peered":"2023-10-07T23:16:28.007883+0000","last_clean":"2023-10-07T23:16:28.007883+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:16:28.007883+0000","last_undegraded":"2023-10-07T23:16:28.007883+0000","last_fullsized":"2023-10-07T23:16:28.007883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999630+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:16:27.999630+0000","last_peered":"2023-10-07T23:16:27.999630+0000","last_clean":"2023-10-07T23:16:27.999630+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:16:27.999630+0000","last_undegraded":"2023-10-07T23:16:27.999630+0000","last_fullsized":"2023-10-07T23:16:27.999630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000842+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:16:28.000842+0000","last_peered":"2023-10-07T23:16:28.000842+0000","last_clean":"2023-10-07T23:16:28.000842+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:16:28.000842+0000","last_undegraded":"2023-10-07T23:16:28.000842+0000","last_fullsized":"2023-10-07T23:16:28.000842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017901+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:16:28.017901+0000","last_peered":"2023-10-07T23:16:28.017901+0000","last_clean":"2023-10-07T23:16:28.017901+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:16:28.017901+0000","last_undegraded":"2023-10-07T23:16:28.017901+0000","last_fullsized":"2023-10-07T23:16:28.017901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011942+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:16:28.011942+0000","last_peered":"2023-10-07T23:16:28.011942+0000","last_clean":"2023-10-07T23:16:28.011942+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:16:28.011942+0000","last_undegraded":"2023-10-07T23:16:28.011942+0000","last_fullsized":"2023-10-07T23:16:28.011942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005454+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:16:28.005454+0000","last_peered":"2023-10-07T23:16:28.005454+0000","last_clean":"2023-10-07T23:16:28.005454+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:16:28.005454+0000","last_undegraded":"2023-10-07T23:16:28.005454+0000","last_fullsized":"2023-10-07T23:16:28.005454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004354+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:16:28.004354+0000","last_peered":"2023-10-07T23:16:28.004354+0000","last_clean":"2023-10-07T23:16:28.004354+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:16:28.004354+0000","last_undegraded":"2023-10-07T23:16:28.004354+0000","last_fullsized":"2023-10-07T23:16:28.004354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009520+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:16:28.009520+0000","last_peered":"2023-10-07T23:16:28.009520+0000","last_clean":"2023-10-07T23:16:28.009520+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.009520+0000","last_undegraded":"2023-10-07T23:16:28.009520+0000","last_fullsized":"2023-10-07T23:16:28.009520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008489+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:16:28.008489+0000","last_peered":"2023-10-07T23:16:28.008489+0000","last_clean":"2023-10-07T23:16:28.008489+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:16:28.008489+0000","last_undegraded":"2023-10-07T23:16:28.008489+0000","last_fullsized":"2023-10-07T23:16:28.008489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006630+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:16:28.006630+0000","last_peered":"2023-10-07T23:16:28.006630+0000","last_clean":"2023-10-07T23:16:28.006630+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:16:28.006630+0000","last_undegraded":"2023-10-07T23:16:28.006630+0000","last_fullsized":"2023-10-07T23:16:28.006630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:21.980549+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:17:21.980549+0000","last_peered":"2023-10-07T23:17:21.980549+0000","last_clean":"2023-10-07T23:17:21.980549+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:17:21.980549+0000","last_undegraded":"2023-10-07T23:17:21.980549+0000","last_fullsized":"2023-10-07T23:17:21.980549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014500+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:16:28.014500+0000","last_peered":"2023-10-07T23:16:28.014500+0000","last_clean":"2023-10-07T23:16:28.014500+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:16:28.014500+0000","last_undegraded":"2023-10-07T23:16:28.014500+0000","last_fullsized":"2023-10-07T23:16:28.014500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999223+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:16:27.999223+0000","last_peered":"2023-10-07T23:16:27.999223+0000","last_clean":"2023-10-07T23:16:27.999223+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:16:27.999223+0000","last_undegraded":"2023-10-07T23:16:27.999223+0000","last_fullsized":"2023-10-07T23:16:27.999223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014597+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:16:28.014597+0000","last_peered":"2023-10-07T23:16:28.014597+0000","last_clean":"2023-10-07T23:16:28.014597+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:16:28.014597+0000","last_undegraded":"2023-10-07T23:16:28.014597+0000","last_fullsized":"2023-10-07T23:16:28.014597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012905+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:16:28.012905+0000","last_peered":"2023-10-07T23:16:28.012905+0000","last_clean":"2023-10-07T23:16:28.012905+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:16:28.012905+0000","last_undegraded":"2023-10-07T23:16:28.012905+0000","last_fullsized":"2023-10-07T23:16:28.012905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999373+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:16:27.999373+0000","last_peered":"2023-10-07T23:16:27.999373+0000","last_clean":"2023-10-07T23:16:27.999373+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:16:27.999373+0000","last_undegraded":"2023-10-07T23:16:27.999373+0000","last_fullsized":"2023-10-07T23:16:27.999373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010037+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:16:28.010037+0000","last_peered":"2023-10-07T23:16:28.010037+0000","last_clean":"2023-10-07T23:16:28.010037+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:16:28.010037+0000","last_undegraded":"2023-10-07T23:16:28.010037+0000","last_fullsized":"2023-10-07T23:16:28.010037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005917+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:16:28.005917+0000","last_peered":"2023-10-07T23:16:28.005917+0000","last_clean":"2023-10-07T23:16:28.005917+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:16:28.005917+0000","last_undegraded":"2023-10-07T23:16:28.005917+0000","last_fullsized":"2023-10-07T23:16:28.005917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999220+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:16:27.999220+0000","last_peered":"2023-10-07T23:16:27.999220+0000","last_clean":"2023-10-07T23:16:27.999220+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:16:27.999220+0000","last_undegraded":"2023-10-07T23:16:27.999220+0000","last_fullsized":"2023-10-07T23:16:27.999220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011048+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:16:28.011048+0000","last_peered":"2023-10-07T23:16:28.011048+0000","last_clean":"2023-10-07T23:16:28.011048+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:16:28.011048+0000","last_undegraded":"2023-10-07T23:16:28.011048+0000","last_fullsized":"2023-10-07T23:16:28.011048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005339+0000","last_change":"2023-10-07T23:15:27.197818+0000","last_active":"2023-10-07T23:16:28.005339+0000","last_peered":"2023-10-07T23:16:28.005339+0000","last_clean":"2023-10-07T23:16:28.005339+0000","last_became_active":"2023-10-07T23:15:27.197522+0000","last_became_peered":"2023-10-07T23:15:27.197522+0000","last_unstale":"2023-10-07T23:16:28.005339+0000","last_undegraded":"2023-10-07T23:16:28.005339+0000","last_fullsized":"2023-10-07T23:16:28.005339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:11.128855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008264+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:16:28.008264+0000","last_peered":"2023-10-07T23:16:28.008264+0000","last_clean":"2023-10-07T23:16:28.008264+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:16:28.008264+0000","last_undegraded":"2023-10-07T23:16:28.008264+0000","last_fullsized":"2023-10-07T23:16:28.008264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975355+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:17:25.975355+0000","last_peered":"2023-10-07T23:17:25.975355+0000","last_clean":"2023-10-07T23:17:25.975355+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:17:25.975355+0000","last_undegraded":"2023-10-07T23:17:25.975355+0000","last_fullsized":"2023-10-07T23:17:25.975355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010625+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:16:28.010625+0000","last_peered":"2023-10-07T23:16:28.010625+0000","last_clean":"2023-10-07T23:16:28.010625+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:16:28.010625+0000","last_undegraded":"2023-10-07T23:16:28.010625+0000","last_fullsized":"2023-10-07T23:16:28.010625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.013091+0000","last_change":"2023-10-07T23:16:29.013091+0000","last_active":"2023-10-07T23:16:29.013091+0000","last_peered":"2023-10-07T23:16:29.013091+0000","last_clean":"2023-10-07T23:16:29.013091+0000","last_became_active":"2023-10-07T23:16:29.012680+0000","last_became_peered":"2023-10-07T23:16:29.012680+0000","last_unstale":"2023-10-07T23:16:29.013091+0000","last_undegraded":"2023-10-07T23:16:29.013091+0000","last_fullsized":"2023-10-07T23:16:29.013091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:13.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.050161+0000","last_change":"2023-10-07T23:16:29.050161+0000","last_active":"2023-10-07T23:16:29.050161+0000","last_peered":"2023-10-07T23:16:29.050161+0000","last_clean":"2023-10-07T23:16:29.050161+0000","last_became_active":"2023-10-07T23:16:29.044559+0000","last_became_peered":"2023-10-07T23:16:29.044559+0000","last_unstale":"2023-10-07T23:16:29.050161+0000","last_undegraded":"2023-10-07T23:16:29.050161+0000","last_fullsized":"2023-10-07T23:16:29.050161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:18.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006444+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:16:28.006444+0000","last_peered":"2023-10-07T23:16:28.006444+0000","last_clean":"2023-10-07T23:16:28.006444+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:16:28.006444+0000","last_undegraded":"2023-10-07T23:16:28.006444+0000","last_fullsized":"2023-10-07T23:16:28.006444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045657+0000","last_change":"2023-10-07T23:16:29.045657+0000","last_active":"2023-10-07T23:16:29.045657+0000","last_peered":"2023-10-07T23:16:29.045657+0000","last_clean":"2023-10-07T23:16:29.045657+0000","last_became_active":"2023-10-07T23:16:29.045268+0000","last_became_peered":"2023-10-07T23:16:29.045268+0000","last_unstale":"2023-10-07T23:16:29.045657+0000","last_undegraded":"2023-10-07T23:16:29.045657+0000","last_fullsized":"2023-10-07T23:16:29.045657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:34.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999559+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:16:27.999559+0000","last_peered":"2023-10-07T23:16:27.999559+0000","last_clean":"2023-10-07T23:16:27.999559+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:16:27.999559+0000","last_undegraded":"2023-10-07T23:16:27.999559+0000","last_fullsized":"2023-10-07T23:16:27.999559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016282+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:16:28.016282+0000","last_peered":"2023-10-07T23:16:28.016282+0000","last_clean":"2023-10-07T23:16:28.016282+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:16:28.016282+0000","last_undegraded":"2023-10-07T23:16:28.016282+0000","last_fullsized":"2023-10-07T23:16:28.016282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012663+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:16:28.012663+0000","last_peered":"2023-10-07T23:16:28.012663+0000","last_clean":"2023-10-07T23:16:28.012663+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:16:28.012663+0000","last_undegraded":"2023-10-07T23:16:28.012663+0000","last_fullsized":"2023-10-07T23:16:28.012663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003839+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:16:28.003839+0000","last_peered":"2023-10-07T23:16:28.003839+0000","last_clean":"2023-10-07T23:16:28.003839+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:16:28.003839+0000","last_undegraded":"2023-10-07T23:16:28.003839+0000","last_fullsized":"2023-10-07T23:16:28.003839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005155+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:16:28.005155+0000","last_peered":"2023-10-07T23:16:28.005155+0000","last_clean":"2023-10-07T23:16:28.005155+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:16:28.005155+0000","last_undegraded":"2023-10-07T23:16:28.005155+0000","last_fullsized":"2023-10-07T23:16:28.005155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008218+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:16:28.008218+0000","last_peered":"2023-10-07T23:16:28.008218+0000","last_clean":"2023-10-07T23:16:28.008218+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:16:28.008218+0000","last_undegraded":"2023-10-07T23:16:28.008218+0000","last_fullsized":"2023-10-07T23:16:28.008218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":207,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975957+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:17:26.975957+0000","last_peered":"2023-10-07T23:17:26.975957+0000","last_clean":"2023-10-07T23:17:26.975957+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:17:26.975957+0000","last_undegraded":"2023-10-07T23:17:26.975957+0000","last_fullsized":"2023-10-07T23:17:26.975957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006164+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:16:28.006164+0000","last_peered":"2023-10-07T23:16:28.006164+0000","last_clean":"2023-10-07T23:16:28.006164+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:16:28.006164+0000","last_undegraded":"2023-10-07T23:16:28.006164+0000","last_fullsized":"2023-10-07T23:16:28.006164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004871+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:16:28.004871+0000","last_peered":"2023-10-07T23:16:28.004871+0000","last_clean":"2023-10-07T23:16:28.004871+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:16:28.004871+0000","last_undegraded":"2023-10-07T23:16:28.004871+0000","last_fullsized":"2023-10-07T23:16:28.004871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007922+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:16:28.007922+0000","last_peered":"2023-10-07T23:16:28.007922+0000","last_clean":"2023-10-07T23:16:28.007922+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:16:28.007922+0000","last_undegraded":"2023-10-07T23:16:28.007922+0000","last_fullsized":"2023-10-07T23:16:28.007922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017142+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:16:28.017142+0000","last_peered":"2023-10-07T23:16:28.017142+0000","last_clean":"2023-10-07T23:16:28.017142+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:16:28.017142+0000","last_undegraded":"2023-10-07T23:16:28.017142+0000","last_fullsized":"2023-10-07T23:16:28.017142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999614+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:16:27.999614+0000","last_peered":"2023-10-07T23:16:27.999614+0000","last_clean":"2023-10-07T23:16:27.999614+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:16:27.999614+0000","last_undegraded":"2023-10-07T23:16:27.999614+0000","last_fullsized":"2023-10-07T23:16:27.999614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005267+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:16:28.005267+0000","last_peered":"2023-10-07T23:16:28.005267+0000","last_clean":"2023-10-07T23:16:28.005267+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:16:28.005267+0000","last_undegraded":"2023-10-07T23:16:28.005267+0000","last_fullsized":"2023-10-07T23:16:28.005267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006062+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:16:28.006062+0000","last_peered":"2023-10-07T23:16:28.006062+0000","last_clean":"2023-10-07T23:16:28.006062+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:16:28.006062+0000","last_undegraded":"2023-10-07T23:16:28.006062+0000","last_fullsized":"2023-10-07T23:16:28.006062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005402+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:16:28.005402+0000","last_peered":"2023-10-07T23:16:28.005402+0000","last_clean":"2023-10-07T23:16:28.005402+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:16:28.005402+0000","last_undegraded":"2023-10-07T23:16:28.005402+0000","last_fullsized":"2023-10-07T23:16:28.005402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'14","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975788+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:17:26.975788+0000","last_peered":"2023-10-07T23:17:26.975788+0000","last_clean":"2023-10-07T23:17:26.975788+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:17:26.975788+0000","last_undegraded":"2023-10-07T23:17:26.975788+0000","last_fullsized":"2023-10-07T23:17:26.975788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":98,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009919+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:16:28.009919+0000","last_peered":"2023-10-07T23:16:28.009919+0000","last_clean":"2023-10-07T23:16:28.009919+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:16:28.009919+0000","last_undegraded":"2023-10-07T23:16:28.009919+0000","last_fullsized":"2023-10-07T23:16:28.009919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005504+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:16:28.005504+0000","last_peered":"2023-10-07T23:16:28.005504+0000","last_clean":"2023-10-07T23:16:28.005504+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:16:28.005504+0000","last_undegraded":"2023-10-07T23:16:28.005504+0000","last_fullsized":"2023-10-07T23:16:28.005504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007745+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:16:28.007745+0000","last_peered":"2023-10-07T23:16:28.007745+0000","last_clean":"2023-10-07T23:16:28.007745+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:16:28.007745+0000","last_undegraded":"2023-10-07T23:16:28.007745+0000","last_fullsized":"2023-10-07T23:16:28.007745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999913+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:16:27.999913+0000","last_peered":"2023-10-07T23:16:27.999913+0000","last_clean":"2023-10-07T23:16:27.999913+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:16:27.999913+0000","last_undegraded":"2023-10-07T23:16:27.999913+0000","last_fullsized":"2023-10-07T23:16:27.999913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002941+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:16:28.002941+0000","last_peered":"2023-10-07T23:16:28.002941+0000","last_clean":"2023-10-07T23:16:28.002941+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:16:28.002941+0000","last_undegraded":"2023-10-07T23:16:28.002941+0000","last_fullsized":"2023-10-07T23:16:28.002941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005850+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:16:28.005850+0000","last_peered":"2023-10-07T23:16:28.005850+0000","last_clean":"2023-10-07T23:16:28.005850+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:16:28.005850+0000","last_undegraded":"2023-10-07T23:16:28.005850+0000","last_fullsized":"2023-10-07T23:16:28.005850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008325+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:16:28.008325+0000","last_peered":"2023-10-07T23:16:28.008325+0000","last_clean":"2023-10-07T23:16:28.008325+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:16:28.008325+0000","last_undegraded":"2023-10-07T23:16:28.008325+0000","last_fullsized":"2023-10-07T23:16:28.008325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044573+0000","last_change":"2023-10-07T23:16:29.044573+0000","last_active":"2023-10-07T23:16:29.044573+0000","last_peered":"2023-10-07T23:16:29.044573+0000","last_clean":"2023-10-07T23:16:29.044573+0000","last_became_active":"2023-10-07T23:16:29.044228+0000","last_became_peered":"2023-10-07T23:16:29.044228+0000","last_unstale":"2023-10-07T23:16:29.044573+0000","last_undegraded":"2023-10-07T23:16:29.044573+0000","last_fullsized":"2023-10-07T23:16:29.044573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:31.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975131+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:17:25.975131+0000","last_peered":"2023-10-07T23:17:25.975131+0000","last_clean":"2023-10-07T23:17:25.975131+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:17:25.975131+0000","last_undegraded":"2023-10-07T23:17:25.975131+0000","last_fullsized":"2023-10-07T23:17:25.975131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":798,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007188+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:16:28.007188+0000","last_peered":"2023-10-07T23:16:28.007188+0000","last_clean":"2023-10-07T23:16:28.007188+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:16:28.007188+0000","last_undegraded":"2023-10-07T23:16:28.007188+0000","last_fullsized":"2023-10-07T23:16:28.007188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003129+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:16:28.003129+0000","last_peered":"2023-10-07T23:16:28.003129+0000","last_clean":"2023-10-07T23:16:28.003129+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:16:28.003129+0000","last_undegraded":"2023-10-07T23:16:28.003129+0000","last_fullsized":"2023-10-07T23:16:28.003129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012034+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:16:28.012034+0000","last_peered":"2023-10-07T23:16:28.012034+0000","last_clean":"2023-10-07T23:16:28.012034+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:16:28.012034+0000","last_undegraded":"2023-10-07T23:16:28.012034+0000","last_fullsized":"2023-10-07T23:16:28.012034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003744+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:16:28.003744+0000","last_peered":"2023-10-07T23:16:28.003744+0000","last_clean":"2023-10-07T23:16:28.003744+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:16:28.003744+0000","last_undegraded":"2023-10-07T23:16:28.003744+0000","last_fullsized":"2023-10-07T23:16:28.003744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003691+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:16:28.003691+0000","last_peered":"2023-10-07T23:16:28.003691+0000","last_clean":"2023-10-07T23:16:28.003691+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:16:28.003691+0000","last_undegraded":"2023-10-07T23:16:28.003691+0000","last_fullsized":"2023-10-07T23:16:28.003691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012511+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:16:28.012511+0000","last_peered":"2023-10-07T23:16:28.012511+0000","last_clean":"2023-10-07T23:16:28.012511+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:16:28.012511+0000","last_undegraded":"2023-10-07T23:16:28.012511+0000","last_fullsized":"2023-10-07T23:16:28.012511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003423+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:16:28.003423+0000","last_peered":"2023-10-07T23:16:28.003423+0000","last_clean":"2023-10-07T23:16:28.003423+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:16:28.003423+0000","last_undegraded":"2023-10-07T23:16:28.003423+0000","last_fullsized":"2023-10-07T23:16:28.003423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010581+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:16:28.010581+0000","last_peered":"2023-10-07T23:16:28.010581+0000","last_clean":"2023-10-07T23:16:28.010581+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:16:28.010581+0000","last_undegraded":"2023-10-07T23:16:28.010581+0000","last_fullsized":"2023-10-07T23:16:28.010581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006472+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:16:28.006472+0000","last_peered":"2023-10-07T23:16:28.006472+0000","last_clean":"2023-10-07T23:16:28.006472+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:16:28.006472+0000","last_undegraded":"2023-10-07T23:16:28.006472+0000","last_fullsized":"2023-10-07T23:16:28.006472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003225+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:16:28.003225+0000","last_peered":"2023-10-07T23:16:28.003225+0000","last_clean":"2023-10-07T23:16:28.003225+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:16:28.003225+0000","last_undegraded":"2023-10-07T23:16:28.003225+0000","last_fullsized":"2023-10-07T23:16:28.003225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001591+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:16:28.001591+0000","last_peered":"2023-10-07T23:16:28.001591+0000","last_clean":"2023-10-07T23:16:28.001591+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:16:28.001591+0000","last_undegraded":"2023-10-07T23:16:28.001591+0000","last_fullsized":"2023-10-07T23:16:28.001591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.986097+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:17:25.986097+0000","last_peered":"2023-10-07T23:17:25.986097+0000","last_clean":"2023-10-07T23:17:25.986097+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:17:25.986097+0000","last_undegraded":"2023-10-07T23:17:25.986097+0000","last_fullsized":"2023-10-07T23:17:25.986097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003060+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:16:28.003060+0000","last_peered":"2023-10-07T23:16:28.003060+0000","last_clean":"2023-10-07T23:16:28.003060+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:16:28.003060+0000","last_undegraded":"2023-10-07T23:16:28.003060+0000","last_fullsized":"2023-10-07T23:16:28.003060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999866+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:16:27.999866+0000","last_peered":"2023-10-07T23:16:27.999866+0000","last_clean":"2023-10-07T23:16:27.999866+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:16:27.999866+0000","last_undegraded":"2023-10-07T23:16:27.999866+0000","last_fullsized":"2023-10-07T23:16:27.999866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004005+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:16:28.004005+0000","last_peered":"2023-10-07T23:16:28.004005+0000","last_clean":"2023-10-07T23:16:28.004005+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:16:28.004005+0000","last_undegraded":"2023-10-07T23:16:28.004005+0000","last_fullsized":"2023-10-07T23:16:28.004005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:22.968999+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:17:22.968999+0000","last_peered":"2023-10-07T23:17:22.968999+0000","last_clean":"2023-10-07T23:17:22.968999+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:17:22.968999+0000","last_undegraded":"2023-10-07T23:17:22.968999+0000","last_fullsized":"2023-10-07T23:17:22.968999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007777+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:16:28.007777+0000","last_peered":"2023-10-07T23:16:28.007777+0000","last_clean":"2023-10-07T23:16:28.007777+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:16:28.007777+0000","last_undegraded":"2023-10-07T23:16:28.007777+0000","last_fullsized":"2023-10-07T23:16:28.007777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008687+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:16:28.008687+0000","last_peered":"2023-10-07T23:16:28.008687+0000","last_clean":"2023-10-07T23:16:28.008687+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:16:28.008687+0000","last_undegraded":"2023-10-07T23:16:28.008687+0000","last_fullsized":"2023-10-07T23:16:28.008687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999386+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:16:27.999386+0000","last_peered":"2023-10-07T23:16:27.999386+0000","last_clean":"2023-10-07T23:16:27.999386+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:16:27.999386+0000","last_undegraded":"2023-10-07T23:16:27.999386+0000","last_fullsized":"2023-10-07T23:16:27.999386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007059+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:16:28.007059+0000","last_peered":"2023-10-07T23:16:28.007059+0000","last_clean":"2023-10-07T23:16:28.007059+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:16:28.007059+0000","last_undegraded":"2023-10-07T23:16:28.007059+0000","last_fullsized":"2023-10-07T23:16:28.007059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005716+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:16:28.005716+0000","last_peered":"2023-10-07T23:16:28.005716+0000","last_clean":"2023-10-07T23:16:28.005716+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:16:28.005716+0000","last_undegraded":"2023-10-07T23:16:28.005716+0000","last_fullsized":"2023-10-07T23:16:28.005716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999364+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:16:27.999364+0000","last_peered":"2023-10-07T23:16:27.999364+0000","last_clean":"2023-10-07T23:16:27.999364+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:16:27.999364+0000","last_undegraded":"2023-10-07T23:16:27.999364+0000","last_fullsized":"2023-10-07T23:16:27.999364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.015552+0000","last_change":"2023-10-07T23:16:29.015552+0000","last_active":"2023-10-07T23:16:29.015552+0000","last_peered":"2023-10-07T23:16:29.015552+0000","last_clean":"2023-10-07T23:16:29.015552+0000","last_became_active":"2023-10-07T23:16:29.010895+0000","last_became_peered":"2023-10-07T23:16:29.010895+0000","last_unstale":"2023-10-07T23:16:29.015552+0000","last_undegraded":"2023-10-07T23:16:29.015552+0000","last_fullsized":"2023-10-07T23:16:29.015552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:40.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.012361+0000","last_change":"2023-10-07T23:16:29.012361+0000","last_active":"2023-10-07T23:16:29.012361+0000","last_peered":"2023-10-07T23:16:29.012361+0000","last_clean":"2023-10-07T23:16:29.012361+0000","last_became_active":"2023-10-07T23:16:29.011918+0000","last_became_peered":"2023-10-07T23:16:29.011918+0000","last_unstale":"2023-10-07T23:16:29.012361+0000","last_undegraded":"2023-10-07T23:16:29.012361+0000","last_fullsized":"2023-10-07T23:16:29.012361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:32.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009934+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:16:28.009934+0000","last_peered":"2023-10-07T23:16:28.009934+0000","last_clean":"2023-10-07T23:16:28.009934+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:16:28.009934+0000","last_undegraded":"2023-10-07T23:16:28.009934+0000","last_fullsized":"2023-10-07T23:16:28.009934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001743+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:16:28.001743+0000","last_peered":"2023-10-07T23:16:28.001743+0000","last_clean":"2023-10-07T23:16:28.001743+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.001743+0000","last_undegraded":"2023-10-07T23:16:28.001743+0000","last_fullsized":"2023-10-07T23:16:28.001743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016218+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:16:28.016218+0000","last_peered":"2023-10-07T23:16:28.016218+0000","last_clean":"2023-10-07T23:16:28.016218+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:16:28.016218+0000","last_undegraded":"2023-10-07T23:16:28.016218+0000","last_fullsized":"2023-10-07T23:16:28.016218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.018993+0000","last_change":"2023-10-07T23:16:29.018993+0000","last_active":"2023-10-07T23:16:29.018993+0000","last_peered":"2023-10-07T23:16:29.018993+0000","last_clean":"2023-10-07T23:16:29.018993+0000","last_became_active":"2023-10-07T23:16:29.008350+0000","last_became_peered":"2023-10-07T23:16:29.008350+0000","last_unstale":"2023-10-07T23:16:29.018993+0000","last_undegraded":"2023-10-07T23:16:29.018993+0000","last_fullsized":"2023-10-07T23:16:29.018993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:16.236418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016364+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:16:28.016364+0000","last_peered":"2023-10-07T23:16:28.016364+0000","last_clean":"2023-10-07T23:16:28.016364+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:16:28.016364+0000","last_undegraded":"2023-10-07T23:16:28.016364+0000","last_fullsized":"2023-10-07T23:16:28.016364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006054+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:16:28.006054+0000","last_peered":"2023-10-07T23:16:28.006054+0000","last_clean":"2023-10-07T23:16:28.006054+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:16:28.006054+0000","last_undegraded":"2023-10-07T23:16:28.006054+0000","last_fullsized":"2023-10-07T23:16:28.006054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999879+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:16:27.999879+0000","last_peered":"2023-10-07T23:16:27.999879+0000","last_clean":"2023-10-07T23:16:27.999879+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:16:27.999879+0000","last_undegraded":"2023-10-07T23:16:27.999879+0000","last_fullsized":"2023-10-07T23:16:27.999879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012811+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:16:28.012811+0000","last_peered":"2023-10-07T23:16:28.012811+0000","last_clean":"2023-10-07T23:16:28.012811+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:16:28.012811+0000","last_undegraded":"2023-10-07T23:16:28.012811+0000","last_fullsized":"2023-10-07T23:16:28.012811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999561+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:16:27.999561+0000","last_peered":"2023-10-07T23:16:27.999561+0000","last_clean":"2023-10-07T23:16:27.999561+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:16:27.999561+0000","last_undegraded":"2023-10-07T23:16:27.999561+0000","last_fullsized":"2023-10-07T23:16:27.999561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005469+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:16:28.005469+0000","last_peered":"2023-10-07T23:16:28.005469+0000","last_clean":"2023-10-07T23:16:28.005469+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:16:28.005469+0000","last_undegraded":"2023-10-07T23:16:28.005469+0000","last_fullsized":"2023-10-07T23:16:28.005469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011194+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:16:28.011194+0000","last_peered":"2023-10-07T23:16:28.011194+0000","last_clean":"2023-10-07T23:16:28.011194+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:16:28.011194+0000","last_undegraded":"2023-10-07T23:16:28.011194+0000","last_fullsized":"2023-10-07T23:16:28.011194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006670+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:16:28.006670+0000","last_peered":"2023-10-07T23:16:28.006670+0000","last_clean":"2023-10-07T23:16:28.006670+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:16:28.006670+0000","last_undegraded":"2023-10-07T23:16:28.006670+0000","last_fullsized":"2023-10-07T23:16:28.006670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008316+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:16:28.008316+0000","last_peered":"2023-10-07T23:16:28.008316+0000","last_clean":"2023-10-07T23:16:28.008316+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:16:28.008316+0000","last_undegraded":"2023-10-07T23:16:28.008316+0000","last_fullsized":"2023-10-07T23:16:28.008316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002767+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:16:28.002767+0000","last_peered":"2023-10-07T23:16:28.002767+0000","last_clean":"2023-10-07T23:16:28.002767+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:16:28.002767+0000","last_undegraded":"2023-10-07T23:16:28.002767+0000","last_fullsized":"2023-10-07T23:16:28.002767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999422+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:16:27.999422+0000","last_peered":"2023-10-07T23:16:27.999422+0000","last_clean":"2023-10-07T23:16:27.999422+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:16:27.999422+0000","last_undegraded":"2023-10-07T23:16:27.999422+0000","last_fullsized":"2023-10-07T23:16:27.999422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006299+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:16:28.006299+0000","last_peered":"2023-10-07T23:16:28.006299+0000","last_clean":"2023-10-07T23:16:28.006299+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:16:28.006299+0000","last_undegraded":"2023-10-07T23:16:28.006299+0000","last_fullsized":"2023-10-07T23:16:28.006299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014270+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:16:28.014270+0000","last_peered":"2023-10-07T23:16:28.014270+0000","last_clean":"2023-10-07T23:16:28.014270+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:16:28.014270+0000","last_undegraded":"2023-10-07T23:16:28.014270+0000","last_fullsized":"2023-10-07T23:16:28.014270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999515+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:16:27.999515+0000","last_peered":"2023-10-07T23:16:27.999515+0000","last_clean":"2023-10-07T23:16:27.999515+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:16:27.999515+0000","last_undegraded":"2023-10-07T23:16:27.999515+0000","last_fullsized":"2023-10-07T23:16:27.999515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.049214+0000","last_change":"2023-10-07T23:16:29.049214+0000","last_active":"2023-10-07T23:16:29.049214+0000","last_peered":"2023-10-07T23:16:29.049214+0000","last_clean":"2023-10-07T23:16:29.049214+0000","last_became_active":"2023-10-07T23:16:29.044778+0000","last_became_peered":"2023-10-07T23:16:29.044778+0000","last_unstale":"2023-10-07T23:16:29.049214+0000","last_undegraded":"2023-10-07T23:16:29.049214+0000","last_fullsized":"2023-10-07T23:16:29.049214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:38.061993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007511+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:16:28.007511+0000","last_peered":"2023-10-07T23:16:28.007511+0000","last_clean":"2023-10-07T23:16:28.007511+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:16:28.007511+0000","last_undegraded":"2023-10-07T23:16:28.007511+0000","last_fullsized":"2023-10-07T23:16:28.007511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005590+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:16:28.005590+0000","last_peered":"2023-10-07T23:16:28.005590+0000","last_clean":"2023-10-07T23:16:28.005590+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:16:28.005590+0000","last_undegraded":"2023-10-07T23:16:28.005590+0000","last_fullsized":"2023-10-07T23:16:28.005590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:22.983295+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:17:22.983295+0000","last_peered":"2023-10-07T23:17:22.983295+0000","last_clean":"2023-10-07T23:17:22.983295+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:17:22.983295+0000","last_undegraded":"2023-10-07T23:17:22.983295+0000","last_fullsized":"2023-10-07T23:17:22.983295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007939+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:16:28.007939+0000","last_peered":"2023-10-07T23:16:28.007939+0000","last_clean":"2023-10-07T23:16:28.007939+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:16:28.007939+0000","last_undegraded":"2023-10-07T23:16:28.007939+0000","last_fullsized":"2023-10-07T23:16:28.007939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012043+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:16:28.012043+0000","last_peered":"2023-10-07T23:16:28.012043+0000","last_clean":"2023-10-07T23:16:28.012043+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:16:28.012043+0000","last_undegraded":"2023-10-07T23:16:28.012043+0000","last_fullsized":"2023-10-07T23:16:28.012043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005371+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:16:28.005371+0000","last_peered":"2023-10-07T23:16:28.005371+0000","last_clean":"2023-10-07T23:16:28.005371+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:16:28.005371+0000","last_undegraded":"2023-10-07T23:16:28.005371+0000","last_fullsized":"2023-10-07T23:16:28.005371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007323+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:16:28.007323+0000","last_peered":"2023-10-07T23:16:28.007323+0000","last_clean":"2023-10-07T23:16:28.007323+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:16:28.007323+0000","last_undegraded":"2023-10-07T23:16:28.007323+0000","last_fullsized":"2023-10-07T23:16:28.007323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003056+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:16:28.003056+0000","last_peered":"2023-10-07T23:16:28.003056+0000","last_clean":"2023-10-07T23:16:28.003056+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:16:28.003056+0000","last_undegraded":"2023-10-07T23:16:28.003056+0000","last_fullsized":"2023-10-07T23:16:28.003056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.051993+0000","last_change":"2023-10-07T23:16:29.051993+0000","last_active":"2023-10-07T23:16:29.051993+0000","last_peered":"2023-10-07T23:16:29.051993+0000","last_clean":"2023-10-07T23:16:29.051993+0000","last_became_active":"2023-10-07T23:16:29.046123+0000","last_became_peered":"2023-10-07T23:16:29.046123+0000","last_unstale":"2023-10-07T23:16:29.051993+0000","last_undegraded":"2023-10-07T23:16:29.051993+0000","last_fullsized":"2023-10-07T23:16:29.051993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:52.174959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008468+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:16:28.008468+0000","last_peered":"2023-10-07T23:16:28.008468+0000","last_clean":"2023-10-07T23:16:28.008468+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:16:28.008468+0000","last_undegraded":"2023-10-07T23:16:28.008468+0000","last_fullsized":"2023-10-07T23:16:28.008468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002713+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:16:28.002713+0000","last_peered":"2023-10-07T23:16:28.002713+0000","last_clean":"2023-10-07T23:16:28.002713+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:16:28.002713+0000","last_undegraded":"2023-10-07T23:16:28.002713+0000","last_fullsized":"2023-10-07T23:16:28.002713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006794+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:16:28.006794+0000","last_peered":"2023-10-07T23:16:28.006794+0000","last_clean":"2023-10-07T23:16:28.006794+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:16:28.006794+0000","last_undegraded":"2023-10-07T23:16:28.006794+0000","last_fullsized":"2023-10-07T23:16:28.006794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006214+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:16:28.006214+0000","last_peered":"2023-10-07T23:16:28.006214+0000","last_clean":"2023-10-07T23:16:28.006214+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:16:28.006214+0000","last_undegraded":"2023-10-07T23:16:28.006214+0000","last_fullsized":"2023-10-07T23:16:28.006214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012237+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:16:28.012237+0000","last_peered":"2023-10-07T23:16:28.012237+0000","last_clean":"2023-10-07T23:16:28.012237+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:16:28.012237+0000","last_undegraded":"2023-10-07T23:16:28.012237+0000","last_fullsized":"2023-10-07T23:16:28.012237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044932+0000","last_change":"2023-10-07T23:16:29.044932+0000","last_active":"2023-10-07T23:16:29.044932+0000","last_peered":"2023-10-07T23:16:29.044932+0000","last_clean":"2023-10-07T23:16:29.044932+0000","last_became_active":"2023-10-07T23:16:29.044454+0000","last_became_peered":"2023-10-07T23:16:29.044454+0000","last_unstale":"2023-10-07T23:16:29.044932+0000","last_undegraded":"2023-10-07T23:16:29.044932+0000","last_fullsized":"2023-10-07T23:16:29.044932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:55.975049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003230+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:16:28.003230+0000","last_peered":"2023-10-07T23:16:28.003230+0000","last_clean":"2023-10-07T23:16:28.003230+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:16:28.003230+0000","last_undegraded":"2023-10-07T23:16:28.003230+0000","last_fullsized":"2023-10-07T23:16:28.003230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008782+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:16:28.008782+0000","last_peered":"2023-10-07T23:16:28.008782+0000","last_clean":"2023-10-07T23:16:28.008782+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:16:28.008782+0000","last_undegraded":"2023-10-07T23:16:28.008782+0000","last_fullsized":"2023-10-07T23:16:28.008782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015307+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:16:28.015307+0000","last_peered":"2023-10-07T23:16:28.015307+0000","last_clean":"2023-10-07T23:16:28.015307+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:16:28.015307+0000","last_undegraded":"2023-10-07T23:16:28.015307+0000","last_fullsized":"2023-10-07T23:16:28.015307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003934+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:16:28.003934+0000","last_peered":"2023-10-07T23:16:28.003934+0000","last_clean":"2023-10-07T23:16:28.003934+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:16:28.003934+0000","last_undegraded":"2023-10-07T23:16:28.003934+0000","last_fullsized":"2023-10-07T23:16:28.003934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008400+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:16:28.008400+0000","last_peered":"2023-10-07T23:16:28.008400+0000","last_clean":"2023-10-07T23:16:28.008400+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:16:28.008400+0000","last_undegraded":"2023-10-07T23:16:28.008400+0000","last_fullsized":"2023-10-07T23:16:28.008400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005732+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:16:28.005732+0000","last_peered":"2023-10-07T23:16:28.005732+0000","last_clean":"2023-10-07T23:16:28.005732+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:16:28.005732+0000","last_undegraded":"2023-10-07T23:16:28.005732+0000","last_fullsized":"2023-10-07T23:16:28.005732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000267+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:16:28.000267+0000","last_peered":"2023-10-07T23:16:28.000267+0000","last_clean":"2023-10-07T23:16:28.000267+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:16:28.000267+0000","last_undegraded":"2023-10-07T23:16:28.000267+0000","last_fullsized":"2023-10-07T23:16:28.000267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003067+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:16:28.003067+0000","last_peered":"2023-10-07T23:16:28.003067+0000","last_clean":"2023-10-07T23:16:28.003067+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:16:28.003067+0000","last_undegraded":"2023-10-07T23:16:28.003067+0000","last_fullsized":"2023-10-07T23:16:28.003067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":954,"num_read_kb":493,"num_write":211,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383715,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":495361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.621},"min":{"1min":0.42999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.755,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.53300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90000000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83299999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.002,"15min":1.002},"last":0.42399999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89200000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0189999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.57399999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.323,"15min":1.323},"last":0.81699999999999995}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.751}]},{"osd":9,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78100000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":0.76000000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0940000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82899999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298908,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446244,"statfs":{"total":95995035648,"available":95688953856,"internally_reserved":0,"allocated":1658880,"data_stored":1089923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.874,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94399999999999995,"5min":1.159,"15min":1.159},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.5,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.141,"5min":1.214,"15min":1.214},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.026,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0149999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76400000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.84299999999999997,"5min":1.171,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.86899999999999999,"15min":0.92700000000000005},"min":{"1min":0.625,"5min":0.502,"15min":0.502},"max":{"1min":0.871,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97299999999999998,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.75},"min":{"1min":0.628,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95899999999999996,"5min":1.179,"15min":1.179},"last":0.78100000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72799999999999998,"15min":0.76000000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.38900000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.054,"15min":1.054},"last":0.501},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68700000000000006,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.85299999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.77300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.278,"15min":1.278},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80800000000000005,"15min":0.84299999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79300000000000004,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.345,"15min":1.345},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82199999999999995,"15min":0.86399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.421,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.85899999999999999,"15min":0.91300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0860000000000001,"5min":1.579,"15min":1.579},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.98099999999999998},"min":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.95399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97699999999999998,"15min":1.0289999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.5880000000000001,"5min":1.603,"15min":1.603},"last":0.623}]},{"osd":8,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.97399999999999998,"15min":1.0389999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2470000000000001,"5min":1.782,"15min":1.782},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.92600000000000005,"15min":0.98499999999999999},"min":{"1min":0.60299999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6850000000000001,"5min":1.956,"15min":1.956},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.97299999999999998,"15min":1.0129999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.647},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.94799999999999995,"15min":1.002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94399999999999995,"5min":1.663,"15min":1.663},"last":0.57899999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.70399999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":1044480,"data_stored":495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57299999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72099999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.93999999999999995}]},{"osd":5,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85699999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.59299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.022,"5min":1.597,"15min":1.597},"last":0.90800000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1,"15min":1},"last":0.98599999999999999}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.80500000000000005,"15min":0.79400000000000004},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4259999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0129999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":1},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76100000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":1.008,"15min":1.008},"last":0.95299999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.92200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298888,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446264,"statfs":{"total":95995035648,"available":95688974336,"internally_reserved":0,"allocated":1638400,"data_stored":1087813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42499999999999999,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.504}]},{"osd":2,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59299999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":1.51,"15min":1.51},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.92800000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0329999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.74199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94599999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58199999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99399999999999999,"5min":1.056,"15min":1.056},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.52500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.002,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.754},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":494596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.62}]},{"osd":1,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.65800000000000003},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.653,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.87}]},{"osd":2,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.83599999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.754},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.82099999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":1.056}]},{"osd":5,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.98099999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.82099999999999995,"15min":0.78700000000000003},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.677,"5min":6.524,"15min":6.524},"last":0.90800000000000003}]},{"osd":8,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.089,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88600000000000001,"15min":0.88900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":2.8780000000000001},"last":1.1850000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.61099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65800000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.2509999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.111,"15min":1.1890000000000001},"min":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.911,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0640000000000001,"15min":1.1299999999999999},"min":{"1min":0.68999999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.294,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.99399999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.03},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.129}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.51800000000000002},"min":{"1min":0.47699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.60999999999999999,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.84299999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.47199999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.433},"min":{"1min":0.33800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65100000000000002,"15min":0.56200000000000006},"min":{"1min":0.498,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.52100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70799999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":1.004,"15min":1.004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89800000000000002,"5min":1.107,"15min":1.107},"last":0.624},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80500000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0069999999999999,"5min":1.131,"15min":1.131},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94699999999999995,"5min":1.075,"15min":1.075},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.752},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.89000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.57099999999999995,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":5.4249999999999998,"5min":5.4249999999999998,"15min":5.4249999999999998},"last":0.65100000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.73499999999999999,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5990000000000002,"5min":6.5990000000000002,"15min":6.5990000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.879,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.76300000000000001,"15min":0.66300000000000003},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.4669999999999996,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.51300000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.81399999999999995,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.88300000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.49399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.6180000000000003,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":5.0609999999999999,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1339999999999999,"5min":1.381,"15min":1.381},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0049999999999999,"15min":0.97999999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":5.5439999999999996,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.68400000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.0029999999999999,"15min":0.94399999999999995},"min":{"1min":0.79300000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":6.6539999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.93600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446836,"statfs":{"total":95995035648,"available":95689560064,"internally_reserved":0,"allocated":1052672,"data_stored":499485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.54100000000000004},"min":{"1min":0.48899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.29899999999999999},"max":{"1min":0.88200000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Oct 7 23:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84999999999999998,"5min":0.871,"15min":0.871},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.746},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.45200000000000001},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42799999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88800000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.57399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85699999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.628,"15min":0.60099999999999998},"min":{"1min":0.59099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56999999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.65100000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.96899999999999997,"5min":1.024,"15min":1.024},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.75700000000000001,"15min":0.71399999999999997},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.03,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.77300000000000002,"15min":0.73099999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.76300000000000001}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":1040384,"data_stored":494675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"min":{"1min":0.34499999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80500000000000005,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.0169999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70899999999999996,"15min":0.629},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92400000000000004,"5min":2.387,"15min":2.387},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0680000000000001,"5min":1.218,"15min":1.218},"last":0.57399999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71699999999999997,"15min":0.64000000000000001},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95699999999999996,"5min":1.278,"15min":1.278},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.68600000000000005},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.01,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.139,"5min":0.80300000000000005,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":6.3940000000000001},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67200000000000004},"min":{"1min":0.46899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.64600000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.89200000000000002,"15min":0.81799999999999995},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":6.4160000000000004,"5min":6.4160000000000004,"15min":6.4160000000000004},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.93400000000000005,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.002,"5min":0.998,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.96099999999999997,"15min":0.99399999999999999},"min":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0269999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.95299999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.82999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":495155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56699999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41299999999999998,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.59199999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.55000000000000004},"min":{"1min":0.51000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.88,"15min":0.88},"last":0.66500000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.498}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.56499999999999995},"min":{"1min":0.52100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.64900000000000002,"15min":0.56899999999999995},"min":{"1min":0.60099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66800000000000004,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":1.004,"15min":1.004},"last":0.57499999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83699999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80400000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.81200000000000006,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.755},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69899999999999995}]},{"osd":10,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.89400000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446256,"statfs":{"total":95995035648,"available":95688966144,"internally_reserved":0,"allocated":1646592,"data_stored":1086978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.6230000000000002,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79800000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.50600000000000001},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71499999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41699999999999998,"15min":0.36899999999999999},"min":{"1min":0.36399999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.63200000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.373},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.38600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":3.827,"15min":3.827},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.55200000000000005},"min":{"1min":0.46400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90800000000000003,"5min":3.903,"15min":3.903},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.54600000000000004},"min":{"1min":0.52200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60799999999999998,"15min":0.55800000000000005},"min":{"1min":0.50800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54500000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.55200000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.749}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.47899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.63400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.747,"15min":0.748},"min":{"1min":0.61899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":1,"15min":1},"last":0.59999999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.65700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.625,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:17:28.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:17:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:28 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/285697511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-07T23:17:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:28 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1171881599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-07T23:17:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:28 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2578655862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:28.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-10-07T23:17:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-07T23:17:28.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:28 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2023-10-07T23:17:28.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3484990622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08104b80 msgr2=0x7fde08107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3484990622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08104b80 0x7fde08107000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde00004ca0 tx=0x7fde00009ca0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3484990622 shutdown_connections 2023-10-07T23:17:28.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3484990622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde08071bc0 0x7fde08070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3484990622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08104b80 0x7fde08107000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3484990622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08070570 0x7fde080709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3484990622 >> 172.21.15.67:0/3484990622 conn(0x7fde080f8460 msgr2=0x7fde080fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3484990622 shutdown_connections 2023-10-07T23:17:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.648+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3484990622 wait complete. 2023-10-07T23:17:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 Processor -- start 2023-10-07T23:17:28.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 -- start start 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde08070570 0x7fde0810f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 0x7fde0810fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08104b80 0x7fde08116f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08117530 con 0x7fde08104b80 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde081176a0 con 0x7fde08070570 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.649+0000 7fde0cf9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08117860 con 0x7fde08071bc0 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 0x7fde0810fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde07fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08104b80 0x7fde08116f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 0x7fde0810fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:47914/0 (socket says 172.21.15.67:47914) 2023-10-07T23:17:28.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 -- 172.21.15.67:0/3203950199 learned_addr learned my addr 172.21.15.67:0/3203950199 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde077fe700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde08070570 0x7fde0810f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 -- 172.21.15.67:0/3203950199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde08070570 msgr2=0x7fde0810f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde08070570 0x7fde0810f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 -- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08104b80 msgr2=0x7fde08116f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08104b80 0x7fde08116f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 -- 172.21.15.67:0/3203950199 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc005040 con 0x7fde08071bc0 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde07fff700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08104b80 0x7fde08116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.650+0000 7fde06ffd700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 0x7fde0810fe90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009730 tx=0x7fddfc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.651+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc007be0 con 0x7fde08071bc0 2023-10-07T23:17:28.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.651+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddfc00f040 con 0x7fde08071bc0 2023-10-07T23:17:28.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.651+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc013bc0 con 0x7fde08071bc0 2023-10-07T23:17:28.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.651+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000048f0 con 0x7fde08071bc0 2023-10-07T23:17:28.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.651+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde08117e70 con 0x7fde08071bc0 2023-10-07T23:17:28.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.652+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde081032d0 con 0x7fde08071bc0 2023-10-07T23:17:28.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.654+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fddfc00ea50 con 0x7fde08071bc0 2023-10-07T23:17:28.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.656+0000 7fddeffff700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fddf009b7f0 0x7fddf009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:28.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.656+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9103+0+0 (secure 0 0 0) 0x7fddfc0c5d20 con 0x7fde08071bc0 2023-10-07T23:17:28.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.657+0000 7fde077fe700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fddf009b7f0 0x7fddf009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:28.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.657+0000 7fde077fe700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fddf009b7f0 0x7fddf009dca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009b80 tx=0x7fddf800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:28.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.658+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fddfc091180 con 0x7fde08071bc0 2023-10-07T23:17:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.855+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde080008d0 con 0x7fddf009b7f0 2023-10-07T23:17:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.868+0000 7fddeffff700 1 -- 172.21.15.67:0/3203950199 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426905 (secure 0 0 0) 0x7fde080008d0 con 0x7fddf009b7f0 2023-10-07T23:17:28.874 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:28.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fddf009b7f0 msgr2=0x7fddf009dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fddf009b7f0 0x7fddf009dca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009b80 tx=0x7fddf800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 msgr2=0x7fde0810fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 0x7fde0810fe90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009730 tx=0x7fddfc009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 shutdown_connections 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde08070570 0x7fde0810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fddf009b7f0 0x7fddf009dca0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde08104b80 0x7fde08116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 --2- 172.21.15.67:0/3203950199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde08071bc0 0x7fde0810fe90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 >> 172.21.15.67:0/3203950199 conn(0x7fde080f8460 msgr2=0x7fde08106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 shutdown_connections 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:28.873+0000 7fde0cf9d700 1 -- 172.21.15.67:0/3203950199 wait complete. 2023-10-07T23:17:28.878 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:17:28.892 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2023-10-07T23:17:28.052053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600431,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1080,"num_read_kb":602,"num_write":392,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3581412,"kb_used_data":13956,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121360412,"statfs":{"total":1151940427776,"available":1148273061888,"internally_reserved":0,"allocated":14290944,"data_stored":7730295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":51,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.101980"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004139+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:16:28.004139+0000","last_peered":"2023-10-07T23:16:28.004139+0000","last_clean":"2023-10-07T23:16:28.004139+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:16:28.004139+0000","last_undegraded":"2023-10-07T23:16:28.004139+0000","last_fullsized":"2023-10-07T23:16:28.004139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012736+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:16:28.012736+0000","last_peered":"2023-10-07T23:16:28.012736+0000","last_clean":"2023-10-07T23:16:28.012736+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:16:28.012736+0000","last_undegraded":"2023-10-07T23:16:28.012736+0000","last_fullsized":"2023-10-07T23:16:28.012736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012507+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:16:28.012507+0000","last_peered":"2023-10-07T23:16:28.012507+0000","last_clean":"2023-10-07T23:16:28.012507+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:16:28.012507+0000","last_undegraded":"2023-10-07T23:16:28.012507+0000","last_fullsized":"2023-10-07T23:16:28.012507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999764+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:16:27.999764+0000","last_peered":"2023-10-07T23:16:27.999764+0000","last_clean":"2023-10-07T23:16:27.999764+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:16:27.999764+0000","last_undegraded":"2023-10-07T23:16:27.999764+0000","last_fullsized":"2023-10-07T23:16:27.999764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011235+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:16:28.011235+0000","last_peered":"2023-10-07T23:16:28.011235+0000","last_clean":"2023-10-07T23:16:28.011235+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:16:28.011235+0000","last_undegraded":"2023-10-07T23:16:28.011235+0000","last_fullsized":"2023-10-07T23:16:28.011235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003946+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:16:28.003946+0000","last_peered":"2023-10-07T23:16:28.003946+0000","last_clean":"2023-10-07T23:16:28.003946+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:16:28.003946+0000","last_undegraded":"2023-10-07T23:16:28.003946+0000","last_fullsized":"2023-10-07T23:16:28.003946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007260+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:16:28.007260+0000","last_peered":"2023-10-07T23:16:28.007260+0000","last_clean":"2023-10-07T23:16:28.007260+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:16:28.007260+0000","last_undegraded":"2023-10-07T23:16:28.007260+0000","last_fullsized":"2023-10-07T23:16:28.007260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015359+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:16:28.015359+0000","last_peered":"2023-10-07T23:16:28.015359+0000","last_clean":"2023-10-07T23:16:28.015359+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:16:28.015359+0000","last_undegraded":"2023-10-07T23:16:28.015359+0000","last_fullsized":"2023-10-07T23:16:28.015359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006337+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:16:28.006337+0000","last_peered":"2023-10-07T23:16:28.006337+0000","last_clean":"2023-10-07T23:16:28.006337+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:16:28.006337+0000","last_undegraded":"2023-10-07T23:16:28.006337+0000","last_fullsized":"2023-10-07T23:16:28.006337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003351+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:16:28.003351+0000","last_peered":"2023-10-07T23:16:28.003351+0000","last_clean":"2023-10-07T23:16:28.003351+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:16:28.003351+0000","last_undegraded":"2023-10-07T23:16:28.003351+0000","last_fullsized":"2023-10-07T23:16:28.003351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007883+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:16:28.007883+0000","last_peered":"2023-10-07T23:16:28.007883+0000","last_clean":"2023-10-07T23:16:28.007883+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:16:28.007883+0000","last_undegraded":"2023-10-07T23:16:28.007883+0000","last_fullsized":"2023-10-07T23:16:28.007883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999630+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:16:27.999630+0000","last_peered":"2023-10-07T23:16:27.999630+0000","last_clean":"2023-10-07T23:16:27.999630+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:16:27.999630+0000","last_undegraded":"2023-10-07T23:16:27.999630+0000","last_fullsized":"2023-10-07T23:16:27.999630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000842+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:16:28.000842+0000","last_peered":"2023-10-07T23:16:28.000842+0000","last_clean":"2023-10-07T23:16:28.000842+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:16:28.000842+0000","last_undegraded":"2023-10-07T23:16:28.000842+0000","last_fullsized":"2023-10-07T23:16:28.000842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017901+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:16:28.017901+0000","last_peered":"2023-10-07T23:16:28.017901+0000","last_clean":"2023-10-07T23:16:28.017901+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:16:28.017901+0000","last_undegraded":"2023-10-07T23:16:28.017901+0000","last_fullsized":"2023-10-07T23:16:28.017901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011942+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:16:28.011942+0000","last_peered":"2023-10-07T23:16:28.011942+0000","last_clean":"2023-10-07T23:16:28.011942+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:16:28.011942+0000","last_undegraded":"2023-10-07T23:16:28.011942+0000","last_fullsized":"2023-10-07T23:16:28.011942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005454+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:16:28.005454+0000","last_peered":"2023-10-07T23:16:28.005454+0000","last_clean":"2023-10-07T23:16:28.005454+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:16:28.005454+0000","last_undegraded":"2023-10-07T23:16:28.005454+0000","last_fullsized":"2023-10-07T23:16:28.005454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004354+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:16:28.004354+0000","last_peered":"2023-10-07T23:16:28.004354+0000","last_clean":"2023-10-07T23:16:28.004354+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:16:28.004354+0000","last_undegraded":"2023-10-07T23:16:28.004354+0000","last_fullsized":"2023-10-07T23:16:28.004354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009520+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:16:28.009520+0000","last_peered":"2023-10-07T23:16:28.009520+0000","last_clean":"2023-10-07T23:16:28.009520+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.009520+0000","last_undegraded":"2023-10-07T23:16:28.009520+0000","last_fullsized":"2023-10-07T23:16:28.009520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008489+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:16:28.008489+0000","last_peered":"2023-10-07T23:16:28.008489+0000","last_clean":"2023-10-07T23:16:28.008489+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:16:28.008489+0000","last_undegraded":"2023-10-07T23:16:28.008489+0000","last_fullsized":"2023-10-07T23:16:28.008489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006630+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:16:28.006630+0000","last_peered":"2023-10-07T23:16:28.006630+0000","last_clean":"2023-10-07T23:16:28.006630+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:16:28.006630+0000","last_undegraded":"2023-10-07T23:16:28.006630+0000","last_fullsized":"2023-10-07T23:16:28.006630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:21.980549+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:17:21.980549+0000","last_peered":"2023-10-07T23:17:21.980549+0000","last_clean":"2023-10-07T23:17:21.980549+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:17:21.980549+0000","last_undegraded":"2023-10-07T23:17:21.980549+0000","last_fullsized":"2023-10-07T23:17:21.980549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014500+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:16:28.014500+0000","last_peered":"2023-10-07T23:16:28.014500+0000","last_clean":"2023-10-07T23:16:28.014500+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:16:28.014500+0000","last_undegraded":"2023-10-07T23:16:28.014500+0000","last_fullsized":"2023-10-07T23:16:28.014500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999223+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:16:27.999223+0000","last_peered":"2023-10-07T23:16:27.999223+0000","last_clean":"2023-10-07T23:16:27.999223+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:16:27.999223+0000","last_undegraded":"2023-10-07T23:16:27.999223+0000","last_fullsized":"2023-10-07T23:16:27.999223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014597+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:16:28.014597+0000","last_peered":"2023-10-07T23:16:28.014597+0000","last_clean":"2023-10-07T23:16:28.014597+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:16:28.014597+0000","last_undegraded":"2023-10-07T23:16:28.014597+0000","last_fullsized":"2023-10-07T23:16:28.014597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012905+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:16:28.012905+0000","last_peered":"2023-10-07T23:16:28.012905+0000","last_clean":"2023-10-07T23:16:28.012905+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:16:28.012905+0000","last_undegraded":"2023-10-07T23:16:28.012905+0000","last_fullsized":"2023-10-07T23:16:28.012905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999373+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:16:27.999373+0000","last_peered":"2023-10-07T23:16:27.999373+0000","last_clean":"2023-10-07T23:16:27.999373+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:16:27.999373+0000","last_undegraded":"2023-10-07T23:16:27.999373+0000","last_fullsized":"2023-10-07T23:16:27.999373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010037+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:16:28.010037+0000","last_peered":"2023-10-07T23:16:28.010037+0000","last_clean":"2023-10-07T23:16:28.010037+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:16:28.010037+0000","last_undegraded":"2023-10-07T23:16:28.010037+0000","last_fullsized":"2023-10-07T23:16:28.010037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005917+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:16:28.005917+0000","last_peered":"2023-10-07T23:16:28.005917+0000","last_clean":"2023-10-07T23:16:28.005917+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:16:28.005917+0000","last_undegraded":"2023-10-07T23:16:28.005917+0000","last_fullsized":"2023-10-07T23:16:28.005917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999220+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:16:27.999220+0000","last_peered":"2023-10-07T23:16:27.999220+0000","last_clean":"2023-10-07T23:16:27.999220+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:16:27.999220+0000","last_undegraded":"2023-10-07T23:16:27.999220+0000","last_fullsized":"2023-10-07T23:16:27.999220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011048+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:16:28.011048+0000","last_peered":"2023-10-07T23:16:28.011048+0000","last_clean":"2023-10-07T23:16:28.011048+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:16:28.011048+0000","last_undegraded":"2023-10-07T23:16:28.011048+0000","last_fullsized":"2023-10-07T23:16:28.011048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005339+0000","last_change":"2023-10-07T23:15:27.197818+0000","last_active":"2023-10-07T23:16:28.005339+0000","last_peered":"2023-10-07T23:16:28.005339+0000","last_clean":"2023-10-07T23:16:28.005339+0000","last_became_active":"2023-10-07T23:15:27.197522+0000","last_became_peered":"2023-10-07T23:15:27.197522+0000","last_unstale":"2023-10-07T23:16:28.005339+0000","last_undegraded":"2023-10-07T23:16:28.005339+0000","last_fullsized":"2023-10-07T23:16:28.005339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:11.128855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008264+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:16:28.008264+0000","last_peered":"2023-10-07T23:16:28.008264+0000","last_clean":"2023-10-07T23:16:28.008264+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:16:28.008264+0000","last_undegraded":"2023-10-07T23:16:28.008264+0000","last_fullsized":"2023-10-07T23:16:28.008264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975355+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:17:25.975355+0000","last_peered":"2023-10-07T23:17:25.975355+0000","last_clean":"2023-10-07T23:17:25.975355+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:17:25.975355+0000","last_undegraded":"2023-10-07T23:17:25.975355+0000","last_fullsized":"2023-10-07T23:17:25.975355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010625+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:16:28.010625+0000","last_peered":"2023-10-07T23:16:28.010625+0000","last_clean":"2023-10-07T23:16:28.010625+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:16:28.010625+0000","last_undegraded":"2023-10-07T23:16:28.010625+0000","last_fullsized":"2023-10-07T23:16:28.010625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.013091+0000","last_change":"2023-10-07T23:16:29.013091+0000","last_active":"2023-10-07T23:16:29.013091+0000","last_peered":"2023-10-07T23:16:29.013091+0000","last_clean":"2023-10-07T23:16:29.013091+0000","last_became_active":"2023-10-07T23:16:29.012680+0000","last_became_peered":"2023-10-07T23:16:29.012680+0000","last_unstale":"2023-10-07T23:16:29.013091+0000","last_undegraded":"2023-10-07T23:16:29.013091+0000","last_fullsized":"2023-10-07T23:16:29.013091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:13.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.050161+0000","last_change":"2023-10-07T23:16:29.050161+0000","last_active":"2023-10-07T23:16:29.050161+0000","last_peered":"2023-10-07T23:16:29.050161+0000","last_clean":"2023-10-07T23:16:29.050161+0000","last_became_active":"2023-10-07T23:16:29.044559+0000","last_became_peered":"2023-10-07T23:16:29.044559+0000","last_unstale":"2023-10-07T23:16:29.050161+0000","last_undegraded":"2023-10-07T23:16:29.050161+0000","last_fullsized":"2023-10-07T23:16:29.050161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:18.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006444+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:16:28.006444+0000","last_peered":"2023-10-07T23:16:28.006444+0000","last_clean":"2023-10-07T23:16:28.006444+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:16:28.006444+0000","last_undegraded":"2023-10-07T23:16:28.006444+0000","last_fullsized":"2023-10-07T23:16:28.006444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045657+0000","last_change":"2023-10-07T23:16:29.045657+0000","last_active":"2023-10-07T23:16:29.045657+0000","last_peered":"2023-10-07T23:16:29.045657+0000","last_clean":"2023-10-07T23:16:29.045657+0000","last_became_active":"2023-10-07T23:16:29.045268+0000","last_became_peered":"2023-10-07T23:16:29.045268+0000","last_unstale":"2023-10-07T23:16:29.045657+0000","last_undegraded":"2023-10-07T23:16:29.045657+0000","last_fullsized":"2023-10-07T23:16:29.045657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:34.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999559+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:16:27.999559+0000","last_peered":"2023-10-07T23:16:27.999559+0000","last_clean":"2023-10-07T23:16:27.999559+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:16:27.999559+0000","last_undegraded":"2023-10-07T23:16:27.999559+0000","last_fullsized":"2023-10-07T23:16:27.999559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016282+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:16:28.016282+0000","last_peered":"2023-10-07T23:16:28.016282+0000","last_clean":"2023-10-07T23:16:28.016282+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:16:28.016282+0000","last_undegraded":"2023-10-07T23:16:28.016282+0000","last_fullsized":"2023-10-07T23:16:28.016282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012663+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:16:28.012663+0000","last_peered":"2023-10-07T23:16:28.012663+0000","last_clean":"2023-10-07T23:16:28.012663+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:16:28.012663+0000","last_undegraded":"2023-10-07T23:16:28.012663+0000","last_fullsized":"2023-10-07T23:16:28.012663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003839+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:16:28.003839+0000","last_peered":"2023-10-07T23:16:28.003839+0000","last_clean":"2023-10-07T23:16:28.003839+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:16:28.003839+0000","last_undegraded":"2023-10-07T23:16:28.003839+0000","last_fullsized":"2023-10-07T23:16:28.003839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005155+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:16:28.005155+0000","last_peered":"2023-10-07T23:16:28.005155+0000","last_clean":"2023-10-07T23:16:28.005155+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:16:28.005155+0000","last_undegraded":"2023-10-07T23:16:28.005155+0000","last_fullsized":"2023-10-07T23:16:28.005155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008218+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:16:28.008218+0000","last_peered":"2023-10-07T23:16:28.008218+0000","last_clean":"2023-10-07T23:16:28.008218+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:16:28.008218+0000","last_undegraded":"2023-10-07T23:16:28.008218+0000","last_fullsized":"2023-10-07T23:16:28.008218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":207,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975957+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:17:26.975957+0000","last_peered":"2023-10-07T23:17:26.975957+0000","last_clean":"2023-10-07T23:17:26.975957+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:17:26.975957+0000","last_undegraded":"2023-10-07T23:17:26.975957+0000","last_fullsized":"2023-10-07T23:17:26.975957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006164+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:16:28.006164+0000","last_peered":"2023-10-07T23:16:28.006164+0000","last_clean":"2023-10-07T23:16:28.006164+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:16:28.006164+0000","last_undegraded":"2023-10-07T23:16:28.006164+0000","last_fullsized":"2023-10-07T23:16:28.006164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004871+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:16:28.004871+0000","last_peered":"2023-10-07T23:16:28.004871+0000","last_clean":"2023-10-07T23:16:28.004871+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:16:28.004871+0000","last_undegraded":"2023-10-07T23:16:28.004871+0000","last_fullsized":"2023-10-07T23:16:28.004871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007922+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:16:28.007922+0000","last_peered":"2023-10-07T23:16:28.007922+0000","last_clean":"2023-10-07T23:16:28.007922+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:16:28.007922+0000","last_undegraded":"2023-10-07T23:16:28.007922+0000","last_fullsized":"2023-10-07T23:16:28.007922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017142+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:16:28.017142+0000","last_peered":"2023-10-07T23:16:28.017142+0000","last_clean":"2023-10-07T23:16:28.017142+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:16:28.017142+0000","last_undegraded":"2023-10-07T23:16:28.017142+0000","last_fullsized":"2023-10-07T23:16:28.017142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999614+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:16:27.999614+0000","last_peered":"2023-10-07T23:16:27.999614+0000","last_clean":"2023-10-07T23:16:27.999614+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:16:27.999614+0000","last_undegraded":"2023-10-07T23:16:27.999614+0000","last_fullsized":"2023-10-07T23:16:27.999614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005267+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:16:28.005267+0000","last_peered":"2023-10-07T23:16:28.005267+0000","last_clean":"2023-10-07T23:16:28.005267+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:16:28.005267+0000","last_undegraded":"2023-10-07T23:16:28.005267+0000","last_fullsized":"2023-10-07T23:16:28.005267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006062+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:16:28.006062+0000","last_peered":"2023-10-07T23:16:28.006062+0000","last_clean":"2023-10-07T23:16:28.006062+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:16:28.006062+0000","last_undegraded":"2023-10-07T23:16:28.006062+0000","last_fullsized":"2023-10-07T23:16:28.006062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005402+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:16:28.005402+0000","last_peered":"2023-10-07T23:16:28.005402+0000","last_clean":"2023-10-07T23:16:28.005402+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:16:28.005402+0000","last_undegraded":"2023-10-07T23:16:28.005402+0000","last_fullsized":"2023-10-07T23:16:28.005402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'14","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975788+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:17:26.975788+0000","last_peered":"2023-10-07T23:17:26.975788+0000","last_clean":"2023-10-07T23:17:26.975788+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:17:26.975788+0000","last_undegraded":"2023-10-07T23:17:26.975788+0000","last_fullsized":"2023-10-07T23:17:26.975788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":98,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009919+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:16:28.009919+0000","last_peered":"2023-10-07T23:16:28.009919+0000","last_clean":"2023-10-07T23:16:28.009919+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:16:28.009919+0000","last_undegraded":"2023-10-07T23:16:28.009919+0000","last_fullsized":"2023-10-07T23:16:28.009919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005504+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:16:28.005504+0000","last_peered":"2023-10-07T23:16:28.005504+0000","last_clean":"2023-10-07T23:16:28.005504+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:16:28.005504+0000","last_undegraded":"2023-10-07T23:16:28.005504+0000","last_fullsized":"2023-10-07T23:16:28.005504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007745+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:16:28.007745+0000","last_peered":"2023-10-07T23:16:28.007745+0000","last_clean":"2023-10-07T23:16:28.007745+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:16:28.007745+0000","last_undegraded":"2023-10-07T23:16:28.007745+0000","last_fullsized":"2023-10-07T23:16:28.007745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999913+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:16:27.999913+0000","last_peered":"2023-10-07T23:16:27.999913+0000","last_clean":"2023-10-07T23:16:27.999913+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:16:27.999913+0000","last_undegraded":"2023-10-07T23:16:27.999913+0000","last_fullsized":"2023-10-07T23:16:27.999913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002941+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:16:28.002941+0000","last_peered":"2023-10-07T23:16:28.002941+0000","last_clean":"2023-10-07T23:16:28.002941+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:16:28.002941+0000","last_undegraded":"2023-10-07T23:16:28.002941+0000","last_fullsized":"2023-10-07T23:16:28.002941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005850+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:16:28.005850+0000","last_peered":"2023-10-07T23:16:28.005850+0000","last_clean":"2023-10-07T23:16:28.005850+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:16:28.005850+0000","last_undegraded":"2023-10-07T23:16:28.005850+0000","last_fullsized":"2023-10-07T23:16:28.005850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008325+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:16:28.008325+0000","last_peered":"2023-10-07T23:16:28.008325+0000","last_clean":"2023-10-07T23:16:28.008325+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:16:28.008325+0000","last_undegraded":"2023-10-07T23:16:28.008325+0000","last_fullsized":"2023-10-07T23:16:28.008325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044573+0000","last_change":"2023-10-07T23:16:29.044573+0000","last_active":"2023-10-07T23:16:29.044573+0000","last_peered":"2023-10-07T23:16:29.044573+0000","last_clean":"2023-10-07T23:16:29.044573+0000","last_became_active":"2023-10-07T23:16:29.044228+0000","last_became_peered":"2023-10-07T23:16:29.044228+0000","last_unstale":"2023-10-07T23:16:29.044573+0000","last_undegraded":"2023-10-07T23:16:29.044573+0000","last_fullsized":"2023-10-07T23:16:29.044573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:31.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975131+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:17:25.975131+0000","last_peered":"2023-10-07T23:17:25.975131+0000","last_clean":"2023-10-07T23:17:25.975131+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:17:25.975131+0000","last_undegraded":"2023-10-07T23:17:25.975131+0000","last_fullsized":"2023-10-07T23:17:25.975131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":798,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007188+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:16:28.007188+0000","last_peered":"2023-10-07T23:16:28.007188+0000","last_clean":"2023-10-07T23:16:28.007188+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:16:28.007188+0000","last_undegraded":"2023-10-07T23:16:28.007188+0000","last_fullsized":"2023-10-07T23:16:28.007188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003129+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:16:28.003129+0000","last_peered":"2023-10-07T23:16:28.003129+0000","last_clean":"2023-10-07T23:16:28.003129+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:16:28.003129+0000","last_undegraded":"2023-10-07T23:16:28.003129+0000","last_fullsized":"2023-10-07T23:16:28.003129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012034+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:16:28.012034+0000","last_peered":"2023-10-07T23:16:28.012034+0000","last_clean":"2023-10-07T23:16:28.012034+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:16:28.012034+0000","last_undegraded":"2023-10-07T23:16:28.012034+0000","last_fullsized":"2023-10-07T23:16:28.012034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003744+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:16:28.003744+0000","last_peered":"2023-10-07T23:16:28.003744+0000","last_clean":"2023-10-07T23:16:28.003744+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:16:28.003744+0000","last_undegraded":"2023-10-07T23:16:28.003744+0000","last_fullsized":"2023-10-07T23:16:28.003744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003691+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:16:28.003691+0000","last_peered":"2023-10-07T23:16:28.003691+0000","last_clean":"2023-10-07T23:16:28.003691+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:16:28.003691+0000","last_undegraded":"2023-10-07T23:16:28.003691+0000","last_fullsized":"2023-10-07T23:16:28.003691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012511+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:16:28.012511+0000","last_peered":"2023-10-07T23:16:28.012511+0000","last_clean":"2023-10-07T23:16:28.012511+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:16:28.012511+0000","last_undegraded":"2023-10-07T23:16:28.012511+0000","last_fullsized":"2023-10-07T23:16:28.012511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003423+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:16:28.003423+0000","last_peered":"2023-10-07T23:16:28.003423+0000","last_clean":"2023-10-07T23:16:28.003423+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:16:28.003423+0000","last_undegraded":"2023-10-07T23:16:28.003423+0000","last_fullsized":"2023-10-07T23:16:28.003423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.010581+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:16:28.010581+0000","last_peered":"2023-10-07T23:16:28.010581+0000","last_clean":"2023-10-07T23:16:28.010581+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:16:28.010581+0000","last_undegraded":"2023-10-07T23:16:28.010581+0000","last_fullsized":"2023-10-07T23:16:28.010581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006472+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:16:28.006472+0000","last_peered":"2023-10-07T23:16:28.006472+0000","last_clean":"2023-10-07T23:16:28.006472+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:16:28.006472+0000","last_undegraded":"2023-10-07T23:16:28.006472+0000","last_fullsized":"2023-10-07T23:16:28.006472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003225+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:16:28.003225+0000","last_peered":"2023-10-07T23:16:28.003225+0000","last_clean":"2023-10-07T23:16:28.003225+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:16:28.003225+0000","last_undegraded":"2023-10-07T23:16:28.003225+0000","last_fullsized":"2023-10-07T23:16:28.003225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001591+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:16:28.001591+0000","last_peered":"2023-10-07T23:16:28.001591+0000","last_clean":"2023-10-07T23:16:28.001591+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:16:28.001591+0000","last_undegraded":"2023-10-07T23:16:28.001591+0000","last_fullsized":"2023-10-07T23:16:28.001591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.986097+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:17:25.986097+0000","last_peered":"2023-10-07T23:17:25.986097+0000","last_clean":"2023-10-07T23:17:25.986097+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:17:25.986097+0000","last_undegraded":"2023-10-07T23:17:25.986097+0000","last_fullsized":"2023-10-07T23:17:25.986097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003060+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:16:28.003060+0000","last_peered":"2023-10-07T23:16:28.003060+0000","last_clean":"2023-10-07T23:16:28.003060+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:16:28.003060+0000","last_undegraded":"2023-10-07T23:16:28.003060+0000","last_fullsized":"2023-10-07T23:16:28.003060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999866+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:16:27.999866+0000","last_peered":"2023-10-07T23:16:27.999866+0000","last_clean":"2023-10-07T23:16:27.999866+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:16:27.999866+0000","last_undegraded":"2023-10-07T23:16:27.999866+0000","last_fullsized":"2023-10-07T23:16:27.999866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004005+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:16:28.004005+0000","last_peered":"2023-10-07T23:16:28.004005+0000","last_clean":"2023-10-07T23:16:28.004005+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:16:28.004005+0000","last_undegraded":"2023-10-07T23:16:28.004005+0000","last_fullsized":"2023-10-07T23:16:28.004005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:22.968999+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:17:22.968999+0000","last_peered":"2023-10-07T23:17:22.968999+0000","last_clean":"2023-10-07T23:17:22.968999+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:17:22.968999+0000","last_undegraded":"2023-10-07T23:17:22.968999+0000","last_fullsized":"2023-10-07T23:17:22.968999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007777+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:16:28.007777+0000","last_peered":"2023-10-07T23:16:28.007777+0000","last_clean":"2023-10-07T23:16:28.007777+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:16:28.007777+0000","last_undegraded":"2023-10-07T23:16:28.007777+0000","last_fullsized":"2023-10-07T23:16:28.007777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008687+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:16:28.008687+0000","last_peered":"2023-10-07T23:16:28.008687+0000","last_clean":"2023-10-07T23:16:28.008687+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:16:28.008687+0000","last_undegraded":"2023-10-07T23:16:28.008687+0000","last_fullsized":"2023-10-07T23:16:28.008687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999386+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:16:27.999386+0000","last_peered":"2023-10-07T23:16:27.999386+0000","last_clean":"2023-10-07T23:16:27.999386+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:16:27.999386+0000","last_undegraded":"2023-10-07T23:16:27.999386+0000","last_fullsized":"2023-10-07T23:16:27.999386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007059+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:16:28.007059+0000","last_peered":"2023-10-07T23:16:28.007059+0000","last_clean":"2023-10-07T23:16:28.007059+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:16:28.007059+0000","last_undegraded":"2023-10-07T23:16:28.007059+0000","last_fullsized":"2023-10-07T23:16:28.007059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005716+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:16:28.005716+0000","last_peered":"2023-10-07T23:16:28.005716+0000","last_clean":"2023-10-07T23:16:28.005716+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:16:28.005716+0000","last_undegraded":"2023-10-07T23:16:28.005716+0000","last_fullsized":"2023-10-07T23:16:28.005716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999364+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:16:27.999364+0000","last_peered":"2023-10-07T23:16:27.999364+0000","last_clean":"2023-10-07T23:16:27.999364+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:16:27.999364+0000","last_undegraded":"2023-10-07T23:16:27.999364+0000","last_fullsized":"2023-10-07T23:16:27.999364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.015552+0000","last_change":"2023-10-07T23:16:29.015552+0000","last_active":"2023-10-07T23:16:29.015552+0000","last_peered":"2023-10-07T23:16:29.015552+0000","last_clean":"2023-10-07T23:16:29.015552+0000","last_became_active":"2023-10-07T23:16:29.010895+0000","last_became_peered":"2023-10-07T23:16:29.010895+0000","last_unstale":"2023-10-07T23:16:29.015552+0000","last_undegraded":"2023-10-07T23:16:29.015552+0000","last_fullsized":"2023-10-07T23:16:29.015552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:40.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.012361+0000","last_change":"2023-10-07T23:16:29.012361+0000","last_active":"2023-10-07T23:16:29.012361+0000","last_peered":"2023-10-07T23:16:29.012361+0000","last_clean":"2023-10-07T23:16:29.012361+0000","last_became_active":"2023-10-07T23:16:29.011918+0000","last_became_peered":"2023-10-07T23:16:29.011918+0000","last_unstale":"2023-10-07T23:16:29.012361+0000","last_undegraded":"2023-10-07T23:16:29.012361+0000","last_fullsized":"2023-10-07T23:16:29.012361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:32.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.009934+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:16:28.009934+0000","last_peered":"2023-10-07T23:16:28.009934+0000","last_clean":"2023-10-07T23:16:28.009934+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:16:28.009934+0000","last_undegraded":"2023-10-07T23:16:28.009934+0000","last_fullsized":"2023-10-07T23:16:28.009934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001743+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:16:28.001743+0000","last_peered":"2023-10-07T23:16:28.001743+0000","last_clean":"2023-10-07T23:16:28.001743+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.001743+0000","last_undegraded":"2023-10-07T23:16:28.001743+0000","last_fullsized":"2023-10-07T23:16:28.001743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016218+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:16:28.016218+0000","last_peered":"2023-10-07T23:16:28.016218+0000","last_clean":"2023-10-07T23:16:28.016218+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:16:28.016218+0000","last_undegraded":"2023-10-07T23:16:28.016218+0000","last_fullsized":"2023-10-07T23:16:28.016218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.018993+0000","last_change":"2023-10-07T23:16:29.018993+0000","last_active":"2023-10-07T23:16:29.018993+0000","last_peered":"2023-10-07T23:16:29.018993+0000","last_clean":"2023-10-07T23:16:29.018993+0000","last_became_active":"2023-10-07T23:16:29.008350+0000","last_became_peered":"2023-10-07T23:16:29.008350+0000","last_unstale":"2023-10-07T23:16:29.018993+0000","last_undegraded":"2023-10-07T23:16:29.018993+0000","last_fullsized":"2023-10-07T23:16:29.018993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:16.236418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016364+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:16:28.016364+0000","last_peered":"2023-10-07T23:16:28.016364+0000","last_clean":"2023-10-07T23:16:28.016364+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:16:28.016364+0000","last_undegraded":"2023-10-07T23:16:28.016364+0000","last_fullsized":"2023-10-07T23:16:28.016364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006054+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:16:28.006054+0000","last_peered":"2023-10-07T23:16:28.006054+0000","last_clean":"2023-10-07T23:16:28.006054+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:16:28.006054+0000","last_undegraded":"2023-10-07T23:16:28.006054+0000","last_fullsized":"2023-10-07T23:16:28.006054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999879+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:16:27.999879+0000","last_peered":"2023-10-07T23:16:27.999879+0000","last_clean":"2023-10-07T23:16:27.999879+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:16:27.999879+0000","last_undegraded":"2023-10-07T23:16:27.999879+0000","last_fullsized":"2023-10-07T23:16:27.999879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012811+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:16:28.012811+0000","last_peered":"2023-10-07T23:16:28.012811+0000","last_clean":"2023-10-07T23:16:28.012811+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:16:28.012811+0000","last_undegraded":"2023-10-07T23:16:28.012811+0000","last_fullsized":"2023-10-07T23:16:28.012811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999561+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:16:27.999561+0000","last_peered":"2023-10-07T23:16:27.999561+0000","last_clean":"2023-10-07T23:16:27.999561+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:16:27.999561+0000","last_undegraded":"2023-10-07T23:16:27.999561+0000","last_fullsized":"2023-10-07T23:16:27.999561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005469+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:16:28.005469+0000","last_peered":"2023-10-07T23:16:28.005469+0000","last_clean":"2023-10-07T23:16:28.005469+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:16:28.005469+0000","last_undegraded":"2023-10-07T23:16:28.005469+0000","last_fullsized":"2023-10-07T23:16:28.005469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011194+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:16:28.011194+0000","last_peered":"2023-10-07T23:16:28.011194+0000","last_clean":"2023-10-07T23:16:28.011194+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:16:28.011194+0000","last_undegraded":"2023-10-07T23:16:28.011194+0000","last_fullsized":"2023-10-07T23:16:28.011194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006670+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:16:28.006670+0000","last_peered":"2023-10-07T23:16:28.006670+0000","last_clean":"2023-10-07T23:16:28.006670+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:16:28.006670+0000","last_undegraded":"2023-10-07T23:16:28.006670+0000","last_fullsized":"2023-10-07T23:16:28.006670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008316+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:16:28.008316+0000","last_peered":"2023-10-07T23:16:28.008316+0000","last_clean":"2023-10-07T23:16:28.008316+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:16:28.008316+0000","last_undegraded":"2023-10-07T23:16:28.008316+0000","last_fullsized":"2023-10-07T23:16:28.008316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002767+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:16:28.002767+0000","last_peered":"2023-10-07T23:16:28.002767+0000","last_clean":"2023-10-07T23:16:28.002767+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:16:28.002767+0000","last_undegraded":"2023-10-07T23:16:28.002767+0000","last_fullsized":"2023-10-07T23:16:28.002767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999422+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:16:27.999422+0000","last_peered":"2023-10-07T23:16:27.999422+0000","last_clean":"2023-10-07T23:16:27.999422+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:16:27.999422+0000","last_undegraded":"2023-10-07T23:16:27.999422+0000","last_fullsized":"2023-10-07T23:16:27.999422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006299+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:16:28.006299+0000","last_peered":"2023-10-07T23:16:28.006299+0000","last_clean":"2023-10-07T23:16:28.006299+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:16:28.006299+0000","last_undegraded":"2023-10-07T23:16:28.006299+0000","last_fullsized":"2023-10-07T23:16:28.006299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014270+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:16:28.014270+0000","last_peered":"2023-10-07T23:16:28.014270+0000","last_clean":"2023-10-07T23:16:28.014270+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:16:28.014270+0000","last_undegraded":"2023-10-07T23:16:28.014270+0000","last_fullsized":"2023-10-07T23:16:28.014270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999515+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:16:27.999515+0000","last_peered":"2023-10-07T23:16:27.999515+0000","last_clean":"2023-10-07T23:16:27.999515+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:16:27.999515+0000","last_undegraded":"2023-10-07T23:16:27.999515+0000","last_fullsized":"2023-10-07T23:16:27.999515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.049214+0000","last_change":"2023-10-07T23:16:29.049214+0000","last_active":"2023-10-07T23:16:29.049214+0000","last_peered":"2023-10-07T23:16:29.049214+0000","last_clean":"2023-10-07T23:16:29.049214+0000","last_became_active":"2023-10-07T23:16:29.044778+0000","last_became_peered":"2023-10-07T23:16:29.044778+0000","last_unstale":"2023-10-07T23:16:29.049214+0000","last_undegraded":"2023-10-07T23:16:29.049214+0000","last_fullsized":"2023-10-07T23:16:29.049214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:38.061993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007511+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:16:28.007511+0000","last_peered":"2023-10-07T23:16:28.007511+0000","last_clean":"2023-10-07T23:16:28.007511+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:16:28.007511+0000","last_undegraded":"2023-10-07T23:16:28.007511+0000","last_fullsized":"2023-10-07T23:16:28.007511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005590+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:16:28.005590+0000","last_peered":"2023-10-07T23:16:28.005590+0000","last_clean":"2023-10-07T23:16:28.005590+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:16:28.005590+0000","last_undegraded":"2023-10-07T23:16:28.005590+0000","last_fullsized":"2023-10-07T23:16:28.005590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:22.983295+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:17:22.983295+0000","last_peered":"2023-10-07T23:17:22.983295+0000","last_clean":"2023-10-07T23:17:22.983295+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:17:22.983295+0000","last_undegraded":"2023-10-07T23:17:22.983295+0000","last_fullsized":"2023-10-07T23:17:22.983295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007939+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:16:28.007939+0000","last_peered":"2023-10-07T23:16:28.007939+0000","last_clean":"2023-10-07T23:16:28.007939+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:16:28.007939+0000","last_undegraded":"2023-10-07T23:16:28.007939+0000","last_fullsized":"2023-10-07T23:16:28.007939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012043+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:16:28.012043+0000","last_peered":"2023-10-07T23:16:28.012043+0000","last_clean":"2023-10-07T23:16:28.012043+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:16:28.012043+0000","last_undegraded":"2023-10-07T23:16:28.012043+0000","last_fullsized":"2023-10-07T23:16:28.012043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005371+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:16:28.005371+0000","last_peered":"2023-10-07T23:16:28.005371+0000","last_clean":"2023-10-07T23:16:28.005371+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:16:28.005371+0000","last_undegraded":"2023-10-07T23:16:28.005371+0000","last_fullsized":"2023-10-07T23:16:28.005371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007323+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:16:28.007323+0000","last_peered":"2023-10-07T23:16:28.007323+0000","last_clean":"2023-10-07T23:16:28.007323+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:16:28.007323+0000","last_undegraded":"2023-10-07T23:16:28.007323+0000","last_fullsized":"2023-10-07T23:16:28.007323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003056+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:16:28.003056+0000","last_peered":"2023-10-07T23:16:28.003056+0000","last_clean":"2023-10-07T23:16:28.003056+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:16:28.003056+0000","last_undegraded":"2023-10-07T23:16:28.003056+0000","last_fullsized":"2023-10-07T23:16:28.003056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.051993+0000","last_change":"2023-10-07T23:16:29.051993+0000","last_active":"2023-10-07T23:16:29.051993+0000","last_peered":"2023-10-07T23:16:29.051993+0000","last_clean":"2023-10-07T23:16:29.051993+0000","last_became_active":"2023-10-07T23:16:29.046123+0000","last_became_peered":"2023-10-07T23:16:29.046123+0000","last_unstale":"2023-10-07T23:16:29.051993+0000","last_undegraded":"2023-10-07T23:16:29.051993+0000","last_fullsized":"2023-10-07T23:16:29.051993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:52.174959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008468+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:16:28.008468+0000","last_peered":"2023-10-07T23:16:28.008468+0000","last_clean":"2023-10-07T23:16:28.008468+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:16:28.008468+0000","last_undegraded":"2023-10-07T23:16:28.008468+0000","last_fullsized":"2023-10-07T23:16:28.008468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002713+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:16:28.002713+0000","last_peered":"2023-10-07T23:16:28.002713+0000","last_clean":"2023-10-07T23:16:28.002713+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:16:28.002713+0000","last_undegraded":"2023-10-07T23:16:28.002713+0000","last_fullsized":"2023-10-07T23:16:28.002713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006794+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:16:28.006794+0000","last_peered":"2023-10-07T23:16:28.006794+0000","last_clean":"2023-10-07T23:16:28.006794+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:16:28.006794+0000","last_undegraded":"2023-10-07T23:16:28.006794+0000","last_fullsized":"2023-10-07T23:16:28.006794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006214+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:16:28.006214+0000","last_peered":"2023-10-07T23:16:28.006214+0000","last_clean":"2023-10-07T23:16:28.006214+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:16:28.006214+0000","last_undegraded":"2023-10-07T23:16:28.006214+0000","last_fullsized":"2023-10-07T23:16:28.006214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012237+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:16:28.012237+0000","last_peered":"2023-10-07T23:16:28.012237+0000","last_clean":"2023-10-07T23:16:28.012237+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:16:28.012237+0000","last_undegraded":"2023-10-07T23:16:28.012237+0000","last_fullsized":"2023-10-07T23:16:28.012237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044932+0000","last_change":"2023-10-07T23:16:29.044932+0000","last_active":"2023-10-07T23:16:29.044932+0000","last_peered":"2023-10-07T23:16:29.044932+0000","last_clean":"2023-10-07T23:16:29.044932+0000","last_became_active":"2023-10-07T23:16:29.044454+0000","last_became_peered":"2023-10-07T23:16:29.044454+0000","last_unstale":"2023-10-07T23:16:29.044932+0000","last_undegraded":"2023-10-07T23:16:29.044932+0000","last_fullsized":"2023-10-07T23:16:29.044932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:55.975049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003230+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:16:28.003230+0000","last_peered":"2023-10-07T23:16:28.003230+0000","last_clean":"2023-10-07T23:16:28.003230+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:16:28.003230+0000","last_undegraded":"2023-10-07T23:16:28.003230+0000","last_fullsized":"2023-10-07T23:16:28.003230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008782+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:16:28.008782+0000","last_peered":"2023-10-07T23:16:28.008782+0000","last_clean":"2023-10-07T23:16:28.008782+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:16:28.008782+0000","last_undegraded":"2023-10-07T23:16:28.008782+0000","last_fullsized":"2023-10-07T23:16:28.008782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015307+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:16:28.015307+0000","last_peered":"2023-10-07T23:16:28.015307+0000","last_clean":"2023-10-07T23:16:28.015307+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:16:28.015307+0000","last_undegraded":"2023-10-07T23:16:28.015307+0000","last_fullsized":"2023-10-07T23:16:28.015307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003934+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:16:28.003934+0000","last_peered":"2023-10-07T23:16:28.003934+0000","last_clean":"2023-10-07T23:16:28.003934+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:16:28.003934+0000","last_undegraded":"2023-10-07T23:16:28.003934+0000","last_fullsized":"2023-10-07T23:16:28.003934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008400+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:16:28.008400+0000","last_peered":"2023-10-07T23:16:28.008400+0000","last_clean":"2023-10-07T23:16:28.008400+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:16:28.008400+0000","last_undegraded":"2023-10-07T23:16:28.008400+0000","last_fullsized":"2023-10-07T23:16:28.008400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005732+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:16:28.005732+0000","last_peered":"2023-10-07T23:16:28.005732+0000","last_clean":"2023-10-07T23:16:28.005732+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:16:28.005732+0000","last_undegraded":"2023-10-07T23:16:28.005732+0000","last_fullsized":"2023-10-07T23:16:28.005732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000267+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:16:28.000267+0000","last_peered":"2023-10-07T23:16:28.000267+0000","last_clean":"2023-10-07T23:16:28.000267+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:16:28.000267+0000","last_undegraded":"2023-10-07T23:16:28.000267+0000","last_fullsized":"2023-10-07T23:16:28.000267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003067+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:16:28.003067+0000","last_peered":"2023-10-07T23:16:28.003067+0000","last_clean":"2023-10-07T23:16:28.003067+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:16:28.003067+0000","last_undegraded":"2023-10-07T23:16:28.003067+0000","last_fullsized":"2023-10-07T23:16:28.003067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":954,"num_read_kb":493,"num_write":211,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383715,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":495361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.621},"min":{"1min":0.42999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.755,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.53300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90000000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83299999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.002,"15min":1.002},"last":0.42399999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89200000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0189999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.57399999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.323,"15min":1.323},"last":0.81699999999999995}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.751}]},{"osd":9,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78100000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":0.76000000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0940000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82899999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298908,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446244,"statfs":{"total":95995035648,"available":95688953856,"internally_reserved":0,"allocated":1658880,"data_stored":1089923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.874,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94399999999999995,"5min":1.159,"15min":1.159},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.5,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.141,"5min":1.214,"15min":1.214},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.026,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0149999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76400000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.84299999999999997,"5min":1.171,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.86899999999999999,"15min":0.92700000000000005},"min":{"1min":0.625,"5min":0.502,"15min":0.502},"max":{"1min":0.871,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97299999999999998,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.75},"min":{"1min":0.628,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95899999999999996,"5min":1.179,"15min":1.179},"last":0.78100000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72799999999999998,"15min":0.76000000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.38900000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.054,"15min":1.054},"last":0.501},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68700000000000006,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.85299999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.77300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.278,"15min":1.278},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80800000000000005,"15min":0.84299999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79300000000000004,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.345,"15min":1.345},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82199999999999995,"15min":0.86399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.421,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.85899999999999999,"15min":0.91300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0860000000000001,"5min":1.579,"15min":1.579},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.98099999999999998},"min":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.95399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97699999999999998,"15min":1.0289999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.5880000000000001,"5min":1.603,"15min":1.603},"last":0.623}]},{"osd":8,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.97399999999999998,"15min":1.0389999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2470000000000001,"5min":1.782,"15min":1.782},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.92600000000000005,"15min":0.98499999999999999},"min":{"1min":0.60299999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6850000000000001,"5min":1.956,"15min":1.956},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.97299999999999998,"15min":1.0129999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.647},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.94799999999999995,"15min":1.002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94399999999999995,"5min":1.663,"15min":1.663},"last":0.57899999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.70399999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":1044480,"data_stored":495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57299999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72099999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.93999999999999995}]},{"osd":5,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85699999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.59299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.022,"5min":1.597,"15min":1.597},"last":0.90800000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1,"15min":1},"last":0.98599999999999999}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.80500000000000005,"15min":0.79400000000000004},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4259999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0129999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":1},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76100000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":1.008,"15min":1.008},"last":0.95299999999999996}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.92200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298888,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446264,"statfs":{"total":95995035648,"available":95688974336,"internally_reserved":0,"allocated":1638400,"data_stored":1087813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42499999999999999,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.504}]},{"osd":2,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59299999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":1.51,"15min":1.51},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.92800000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0329999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.74199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94599999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58199999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99399999999999999,"5min":1.056,"15min":1.056},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.52500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.002,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.754},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":494596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.62}]},{"osd":1,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.65800000000000003},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.653,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.87}]},{"osd":2,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.83599999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.754},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.82099999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":1.056}]},{"osd":5,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.98099999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.82099999999999995,"15min":0.78700000000000003},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.677,"5min":6.524,"15min":6.524},"last":0.90800000000000003}]},{"osd":8,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.089,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88600000000000001,"15min":0.88900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":2.8780000000000001},"last":1.1850000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.61099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65800000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.2509999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.111,"15min":1.1890000000000001},"min":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.911,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0640000000000001,"15min":1.1299999999999999},"min":{"1min":0.68999999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.294,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.99399999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.03},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.129}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.51800000000000002},"min":{"1min":0.47699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.60999999999999999,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.84299999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.47199999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.433},"min":{"1min":0.33800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65100000000000002,"15min":0.56200000000000006},"min":{"1min":0.498,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.52100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70799999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":1.004,"15min":1.004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89800000000000002,"5min":1.107,"15min":1.107},"last":0.624},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80500000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0069999999999999,"5min":1.131,"15min":1.131},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94699999999999995,"5min":1.075,"15min":1.075},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.752},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.89000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.57099999999999995,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":5.4249999999999998,"5min":5.4249999999999998,"15min":5.4249999999999998},"last":0.65100000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.73499999999999999,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5990000000000002,"5min":6.5990000000000002,"15min":6.5990000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.879,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.76300000000000001,"15min":0.66300000000000003},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.4669999999999996,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.51300000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.81399999999999995,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.88300000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.49399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.6180000000000003,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":5.0609999999999999,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1339999999999999,"5min":1.381,"15min":1.381},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0049999999999999,"15min":0.97999999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":5.5439999999999996,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.68400000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.0029999999999999,"15min":0.94399999999999995},"min":{"1min":0.79300000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":6.6539999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.93600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446836,"statfs":{"total":95995035648,"available":95689560064,"internally_reserved":0,"allocated":1052672,"data_stored":499485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.54100000000000004},"min":{"1min":0.48899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.29899999999999999},"max":{"1min":0.88200000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Oct 7 23:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84999999999999998,"5min":0.871,"15min":0.871},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.746},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.45200000000000001},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42799999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88800000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.57399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85699999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.628,"15min":0.60099999999999998},"min":{"1min":0.59099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56999999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.65100000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.96899999999999997,"5min":1.024,"15min":1.024},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.75700000000000001,"15min":0.71399999999999997},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.03,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.77300000000000002,"15min":0.73099999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.76300000000000001}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":1040384,"data_stored":494675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"min":{"1min":0.34499999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80500000000000005,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.0169999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70899999999999996,"15min":0.629},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92400000000000004,"5min":2.387,"15min":2.387},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0680000000000001,"5min":1.218,"15min":1.218},"last":0.57399999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71699999999999997,"15min":0.64000000000000001},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95699999999999996,"5min":1.278,"15min":1.278},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.68600000000000005},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.01,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.139,"5min":0.80300000000000005,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":6.3940000000000001},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67200000000000004},"min":{"1min":0.46899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.64600000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.89200000000000002,"15min":0.81799999999999995},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":6.4160000000000004,"5min":6.4160000000000004,"15min":6.4160000000000004},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.93400000000000005,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.002,"5min":0.998,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.96099999999999997,"15min":0.99399999999999999},"min":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0269999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.95299999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.82999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":495155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56699999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41299999999999998,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.59199999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.55000000000000004},"min":{"1min":0.51000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.88,"15min":0.88},"last":0.66500000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.498}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.56499999999999995},"min":{"1min":0.52100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.64900000000000002,"15min":0.56899999999999995},"min":{"1min":0.60099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66800000000000004,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":1.004,"15min":1.004},"last":0.57499999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83699999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80400000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.81200000000000006,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.755},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69899999999999995}]},{"osd":10,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.89400000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446256,"statfs":{"total":95995035648,"available":95688966144,"internally_reserved":0,"allocated":1646592,"data_stored":1086978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.6230000000000002,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79800000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.50600000000000001},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71499999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41699999999999998,"15min":0.36899999999999999},"min":{"1min":0.36399999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.63200000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.373},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.38600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":3.827,"15min":3.827},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.55200000000000005},"min":{"1min":0.46400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90800000000000003,"5min":3.903,"15min":3.903},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.54600000000000004},"min":{"1min":0.52200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60799999999999998,"15min":0.55800000000000005},"min":{"1min":0.50800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54500000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.55200000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.749}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.47899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.63400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.747,"15min":0.748},"min":{"1min":0.61899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":1,"15min":1},"last":0.59999999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.65700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.625,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:17:28.902 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-07T23:17:28.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:17:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 -- 172.21.15.67:0/882923784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fcce0 msgr2=0x7fdfe00fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/882923784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fcce0 0x7fdfe00fd0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0004900 tx=0x7fdfd0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 -- 172.21.15.67:0/882923784 shutdown_connections 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/882923784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfe00fe880 0x7fdfe0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/882923784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fcce0 0x7fdfe00fd0f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/882923784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe00fdef0 0x7fdfe00fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.125+0000 7fdfe49fe700 1 -- 172.21.15.67:0/882923784 >> 172.21.15.67:0/882923784 conn(0x7fdfe00f8460 msgr2=0x7fdfe00fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.126+0000 7fdfe49fe700 1 -- 172.21.15.67:0/882923784 shutdown_connections 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.126+0000 7fdfe49fe700 1 -- 172.21.15.67:0/882923784 wait complete. 2023-10-07T23:17:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.126+0000 7fdfe49fe700 1 Processor -- start 2023-10-07T23:17:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.126+0000 7fdfe49fe700 1 -- start start 2023-10-07T23:17:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfe49fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe00fcce0 0x7fdfe0071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfe49fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 0x7fdfe006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfe49fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfe00fe880 0x7fdfe00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfe49fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0070c40 con 0x7fdfe00fe880 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfe49fe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0070db0 con 0x7fdfe00fcce0 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfe49fe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0117a90 con 0x7fdfe00fdef0 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 0x7fdfe006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfdf59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfe00fe880 0x7fdfe00704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 0x7fdfe006ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.67:47922/0 (socket says 172.21.15.67:47922) 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 -- 172.21.15.67:0/689781913 learned_addr learned my addr 172.21.15.67:0/689781913 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfded9d700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe00fcce0 0x7fdfe0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 -- 172.21.15.67:0/689781913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe00fcce0 msgr2=0x7fdfe0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe00fcce0 0x7fdfe0071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 -- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfe00fe880 msgr2=0x7fdfe00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfe00fe880 0x7fdfe00704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.127+0000 7fdfde59c700 1 -- 172.21.15.67:0/689781913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd0005040 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.128+0000 7fdfde59c700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 0x7fdfe006ff90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400ebc0 tx=0x7fdfd400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.128+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4007cc0 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.128+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfd400f040 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.128+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400c880 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.128+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe0117ce0 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.128+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe0118200 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.129+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe0108520 con 0x7fdfe00fdef0 2023-10-07T23:17:29.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.131+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fdfd400ccb0 con 0x7fdfe00fdef0 2023-10-07T23:17:29.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.132+0000 7fdfc77fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdfc809b7f0 0x7fdfc809dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.132+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9103+0+0 (secure 0 0 0) 0x7fdfd40c4da0 con 0x7fdfe00fdef0 2023-10-07T23:17:29.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.132+0000 7fdfded9d700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdfc809b7f0 0x7fdfc809dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.133+0000 7fdfded9d700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdfc809b7f0 0x7fdfc809dca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00047d0 tx=0x7fdfd0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:29.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.134+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fdfd4090200 con 0x7fdfe00fdef0 2023-10-07T23:17:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:29 smithi067 ceph-mon[82723]: pgmap v351: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-07T23:17:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-10-07T23:17:29.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-07T23:17:29.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:29 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-10-07T23:17:29.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:29 smithi067 ceph-mon[82723]: osdmap e92: 12 total, 12 up, 12 in 2023-10-07T23:17:29.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:29 smithi067 ceph-mon[82723]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:29.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.337+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfe01183f0 con 0x7fdfc809b7f0 2023-10-07T23:17:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.350+0000 7fdfc77fe700 1 -- 172.21.15.67:0/689781913 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426837 (secure 0 0 0) 0x7fdfe01183f0 con 0x7fdfc809b7f0 2023-10-07T23:17:29.356 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdfc809b7f0 msgr2=0x7fdfc809dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdfc809b7f0 0x7fdfc809dca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00047d0 tx=0x7fdfd0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 msgr2=0x7fdfe006ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 0x7fdfe006ff90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400ebc0 tx=0x7fdfd400b040 comp rx=0 tx=0).stop 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 shutdown_connections 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe00fcce0 0x7fdfe0071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fdfc809b7f0 0x7fdfc809dca0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfe00fe880 0x7fdfe00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 --2- 172.21.15.67:0/689781913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe00fdef0 0x7fdfe006ff90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 >> 172.21.15.67:0/689781913 conn(0x7fdfe00f8460 msgr2=0x7fdfe00fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 shutdown_connections 2023-10-07T23:17:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.355+0000 7fdfe49fe700 1 -- 172.21.15.67:0/689781913 wait complete. 2023-10-07T23:17:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:17:29.373 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-10-07T23:17:29.056253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600431,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1090,"num_read_kb":612,"num_write":392,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3581412,"kb_used_data":13956,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121360412,"statfs":{"total":1151940427776,"available":1148273061888,"internally_reserved":0,"allocated":14290944,"data_stored":7730295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":51,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.105690"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004139+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:16:28.004139+0000","last_peered":"2023-10-07T23:16:28.004139+0000","last_clean":"2023-10-07T23:16:28.004139+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:16:28.004139+0000","last_undegraded":"2023-10-07T23:16:28.004139+0000","last_fullsized":"2023-10-07T23:16:28.004139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012736+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:16:28.012736+0000","last_peered":"2023-10-07T23:16:28.012736+0000","last_clean":"2023-10-07T23:16:28.012736+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:16:28.012736+0000","last_undegraded":"2023-10-07T23:16:28.012736+0000","last_fullsized":"2023-10-07T23:16:28.012736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012507+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:16:28.012507+0000","last_peered":"2023-10-07T23:16:28.012507+0000","last_clean":"2023-10-07T23:16:28.012507+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:16:28.012507+0000","last_undegraded":"2023-10-07T23:16:28.012507+0000","last_fullsized":"2023-10-07T23:16:28.012507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999764+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:16:27.999764+0000","last_peered":"2023-10-07T23:16:27.999764+0000","last_clean":"2023-10-07T23:16:27.999764+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:16:27.999764+0000","last_undegraded":"2023-10-07T23:16:27.999764+0000","last_fullsized":"2023-10-07T23:16:27.999764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.008822+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:16:29.008822+0000","last_peered":"2023-10-07T23:16:29.008822+0000","last_clean":"2023-10-07T23:16:29.008822+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:16:29.008822+0000","last_undegraded":"2023-10-07T23:16:29.008822+0000","last_fullsized":"2023-10-07T23:16:29.008822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003946+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:16:28.003946+0000","last_peered":"2023-10-07T23:16:28.003946+0000","last_clean":"2023-10-07T23:16:28.003946+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:16:28.003946+0000","last_undegraded":"2023-10-07T23:16:28.003946+0000","last_fullsized":"2023-10-07T23:16:28.003946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007260+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:16:28.007260+0000","last_peered":"2023-10-07T23:16:28.007260+0000","last_clean":"2023-10-07T23:16:28.007260+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:16:28.007260+0000","last_undegraded":"2023-10-07T23:16:28.007260+0000","last_fullsized":"2023-10-07T23:16:28.007260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015359+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:16:28.015359+0000","last_peered":"2023-10-07T23:16:28.015359+0000","last_clean":"2023-10-07T23:16:28.015359+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:16:28.015359+0000","last_undegraded":"2023-10-07T23:16:28.015359+0000","last_fullsized":"2023-10-07T23:16:28.015359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006337+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:16:28.006337+0000","last_peered":"2023-10-07T23:16:28.006337+0000","last_clean":"2023-10-07T23:16:28.006337+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:16:28.006337+0000","last_undegraded":"2023-10-07T23:16:28.006337+0000","last_fullsized":"2023-10-07T23:16:28.006337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003351+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:16:28.003351+0000","last_peered":"2023-10-07T23:16:28.003351+0000","last_clean":"2023-10-07T23:16:28.003351+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:16:28.003351+0000","last_undegraded":"2023-10-07T23:16:28.003351+0000","last_fullsized":"2023-10-07T23:16:28.003351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007883+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:16:28.007883+0000","last_peered":"2023-10-07T23:16:28.007883+0000","last_clean":"2023-10-07T23:16:28.007883+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:16:28.007883+0000","last_undegraded":"2023-10-07T23:16:28.007883+0000","last_fullsized":"2023-10-07T23:16:28.007883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999630+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:16:27.999630+0000","last_peered":"2023-10-07T23:16:27.999630+0000","last_clean":"2023-10-07T23:16:27.999630+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:16:27.999630+0000","last_undegraded":"2023-10-07T23:16:27.999630+0000","last_fullsized":"2023-10-07T23:16:27.999630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000842+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:16:28.000842+0000","last_peered":"2023-10-07T23:16:28.000842+0000","last_clean":"2023-10-07T23:16:28.000842+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:16:28.000842+0000","last_undegraded":"2023-10-07T23:16:28.000842+0000","last_fullsized":"2023-10-07T23:16:28.000842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017901+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:16:28.017901+0000","last_peered":"2023-10-07T23:16:28.017901+0000","last_clean":"2023-10-07T23:16:28.017901+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:16:28.017901+0000","last_undegraded":"2023-10-07T23:16:28.017901+0000","last_fullsized":"2023-10-07T23:16:28.017901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011942+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:16:28.011942+0000","last_peered":"2023-10-07T23:16:28.011942+0000","last_clean":"2023-10-07T23:16:28.011942+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:16:28.011942+0000","last_undegraded":"2023-10-07T23:16:28.011942+0000","last_fullsized":"2023-10-07T23:16:28.011942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005454+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:16:28.005454+0000","last_peered":"2023-10-07T23:16:28.005454+0000","last_clean":"2023-10-07T23:16:28.005454+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:16:28.005454+0000","last_undegraded":"2023-10-07T23:16:28.005454+0000","last_fullsized":"2023-10-07T23:16:28.005454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004354+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:16:28.004354+0000","last_peered":"2023-10-07T23:16:28.004354+0000","last_clean":"2023-10-07T23:16:28.004354+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:16:28.004354+0000","last_undegraded":"2023-10-07T23:16:28.004354+0000","last_fullsized":"2023-10-07T23:16:28.004354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.002524+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:16:29.002524+0000","last_peered":"2023-10-07T23:16:29.002524+0000","last_clean":"2023-10-07T23:16:29.002524+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:29.002524+0000","last_undegraded":"2023-10-07T23:16:29.002524+0000","last_fullsized":"2023-10-07T23:16:29.002524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008489+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:16:28.008489+0000","last_peered":"2023-10-07T23:16:28.008489+0000","last_clean":"2023-10-07T23:16:28.008489+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:16:28.008489+0000","last_undegraded":"2023-10-07T23:16:28.008489+0000","last_fullsized":"2023-10-07T23:16:28.008489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006630+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:16:28.006630+0000","last_peered":"2023-10-07T23:16:28.006630+0000","last_clean":"2023-10-07T23:16:28.006630+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:16:28.006630+0000","last_undegraded":"2023-10-07T23:16:28.006630+0000","last_fullsized":"2023-10-07T23:16:28.006630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":125,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:27.987892+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:17:27.987892+0000","last_peered":"2023-10-07T23:17:27.987892+0000","last_clean":"2023-10-07T23:17:27.987892+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:17:27.987892+0000","last_undegraded":"2023-10-07T23:17:27.987892+0000","last_fullsized":"2023-10-07T23:17:27.987892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014500+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:16:28.014500+0000","last_peered":"2023-10-07T23:16:28.014500+0000","last_clean":"2023-10-07T23:16:28.014500+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:16:28.014500+0000","last_undegraded":"2023-10-07T23:16:28.014500+0000","last_fullsized":"2023-10-07T23:16:28.014500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999223+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:16:27.999223+0000","last_peered":"2023-10-07T23:16:27.999223+0000","last_clean":"2023-10-07T23:16:27.999223+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:16:27.999223+0000","last_undegraded":"2023-10-07T23:16:27.999223+0000","last_fullsized":"2023-10-07T23:16:27.999223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014597+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:16:28.014597+0000","last_peered":"2023-10-07T23:16:28.014597+0000","last_clean":"2023-10-07T23:16:28.014597+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:16:28.014597+0000","last_undegraded":"2023-10-07T23:16:28.014597+0000","last_fullsized":"2023-10-07T23:16:28.014597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012905+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:16:28.012905+0000","last_peered":"2023-10-07T23:16:28.012905+0000","last_clean":"2023-10-07T23:16:28.012905+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:16:28.012905+0000","last_undegraded":"2023-10-07T23:16:28.012905+0000","last_fullsized":"2023-10-07T23:16:28.012905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999373+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:16:27.999373+0000","last_peered":"2023-10-07T23:16:27.999373+0000","last_clean":"2023-10-07T23:16:27.999373+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:16:27.999373+0000","last_undegraded":"2023-10-07T23:16:27.999373+0000","last_fullsized":"2023-10-07T23:16:27.999373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.007402+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:16:29.007402+0000","last_peered":"2023-10-07T23:16:29.007402+0000","last_clean":"2023-10-07T23:16:29.007402+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:16:29.007402+0000","last_undegraded":"2023-10-07T23:16:29.007402+0000","last_fullsized":"2023-10-07T23:16:29.007402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005917+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:16:28.005917+0000","last_peered":"2023-10-07T23:16:28.005917+0000","last_clean":"2023-10-07T23:16:28.005917+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:16:28.005917+0000","last_undegraded":"2023-10-07T23:16:28.005917+0000","last_fullsized":"2023-10-07T23:16:28.005917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999220+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:16:27.999220+0000","last_peered":"2023-10-07T23:16:27.999220+0000","last_clean":"2023-10-07T23:16:27.999220+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:16:27.999220+0000","last_undegraded":"2023-10-07T23:16:27.999220+0000","last_fullsized":"2023-10-07T23:16:27.999220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.006020+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:16:29.006020+0000","last_peered":"2023-10-07T23:16:29.006020+0000","last_clean":"2023-10-07T23:16:29.006020+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:16:29.006020+0000","last_undegraded":"2023-10-07T23:16:29.006020+0000","last_fullsized":"2023-10-07T23:16:29.006020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"peering","last_fresh":"2023-10-07T23:17:28.058734+0000","last_change":"2023-10-07T23:17:28.058734+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:17:28.058734+0000","last_undegraded":"2023-10-07T23:17:28.058734+0000","last_fullsized":"2023-10-07T23:17:28.058734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:45:18.855595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008264+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:16:28.008264+0000","last_peered":"2023-10-07T23:16:28.008264+0000","last_clean":"2023-10-07T23:16:28.008264+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:16:28.008264+0000","last_undegraded":"2023-10-07T23:16:28.008264+0000","last_fullsized":"2023-10-07T23:16:28.008264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975355+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:17:25.975355+0000","last_peered":"2023-10-07T23:17:25.975355+0000","last_clean":"2023-10-07T23:17:25.975355+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:17:25.975355+0000","last_undegraded":"2023-10-07T23:17:25.975355+0000","last_fullsized":"2023-10-07T23:17:25.975355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005663+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:16:29.005663+0000","last_peered":"2023-10-07T23:16:29.005663+0000","last_clean":"2023-10-07T23:16:29.005663+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:16:29.005663+0000","last_undegraded":"2023-10-07T23:16:29.005663+0000","last_fullsized":"2023-10-07T23:16:29.005663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.013091+0000","last_change":"2023-10-07T23:16:29.013091+0000","last_active":"2023-10-07T23:16:29.013091+0000","last_peered":"2023-10-07T23:16:29.013091+0000","last_clean":"2023-10-07T23:16:29.013091+0000","last_became_active":"2023-10-07T23:16:29.012680+0000","last_became_peered":"2023-10-07T23:16:29.012680+0000","last_unstale":"2023-10-07T23:16:29.013091+0000","last_undegraded":"2023-10-07T23:16:29.013091+0000","last_fullsized":"2023-10-07T23:16:29.013091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:13.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.050161+0000","last_change":"2023-10-07T23:16:29.050161+0000","last_active":"2023-10-07T23:16:29.050161+0000","last_peered":"2023-10-07T23:16:29.050161+0000","last_clean":"2023-10-07T23:16:29.050161+0000","last_became_active":"2023-10-07T23:16:29.044559+0000","last_became_peered":"2023-10-07T23:16:29.044559+0000","last_unstale":"2023-10-07T23:16:29.050161+0000","last_undegraded":"2023-10-07T23:16:29.050161+0000","last_fullsized":"2023-10-07T23:16:29.050161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:18.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006444+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:16:28.006444+0000","last_peered":"2023-10-07T23:16:28.006444+0000","last_clean":"2023-10-07T23:16:28.006444+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:16:28.006444+0000","last_undegraded":"2023-10-07T23:16:28.006444+0000","last_fullsized":"2023-10-07T23:16:28.006444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045657+0000","last_change":"2023-10-07T23:16:29.045657+0000","last_active":"2023-10-07T23:16:29.045657+0000","last_peered":"2023-10-07T23:16:29.045657+0000","last_clean":"2023-10-07T23:16:29.045657+0000","last_became_active":"2023-10-07T23:16:29.045268+0000","last_became_peered":"2023-10-07T23:16:29.045268+0000","last_unstale":"2023-10-07T23:16:29.045657+0000","last_undegraded":"2023-10-07T23:16:29.045657+0000","last_fullsized":"2023-10-07T23:16:29.045657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:34.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999559+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:16:27.999559+0000","last_peered":"2023-10-07T23:16:27.999559+0000","last_clean":"2023-10-07T23:16:27.999559+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:16:27.999559+0000","last_undegraded":"2023-10-07T23:16:27.999559+0000","last_fullsized":"2023-10-07T23:16:27.999559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016282+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:16:28.016282+0000","last_peered":"2023-10-07T23:16:28.016282+0000","last_clean":"2023-10-07T23:16:28.016282+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:16:28.016282+0000","last_undegraded":"2023-10-07T23:16:28.016282+0000","last_fullsized":"2023-10-07T23:16:28.016282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012663+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:16:28.012663+0000","last_peered":"2023-10-07T23:16:28.012663+0000","last_clean":"2023-10-07T23:16:28.012663+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:16:28.012663+0000","last_undegraded":"2023-10-07T23:16:28.012663+0000","last_fullsized":"2023-10-07T23:16:28.012663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003839+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:16:28.003839+0000","last_peered":"2023-10-07T23:16:28.003839+0000","last_clean":"2023-10-07T23:16:28.003839+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:16:28.003839+0000","last_undegraded":"2023-10-07T23:16:28.003839+0000","last_fullsized":"2023-10-07T23:16:28.003839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.000644+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:16:29.000644+0000","last_peered":"2023-10-07T23:16:29.000644+0000","last_clean":"2023-10-07T23:16:29.000644+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:16:29.000644+0000","last_undegraded":"2023-10-07T23:16:29.000644+0000","last_fullsized":"2023-10-07T23:16:29.000644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008218+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:16:28.008218+0000","last_peered":"2023-10-07T23:16:28.008218+0000","last_clean":"2023-10-07T23:16:28.008218+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:16:28.008218+0000","last_undegraded":"2023-10-07T23:16:28.008218+0000","last_fullsized":"2023-10-07T23:16:28.008218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":207,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975957+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:17:26.975957+0000","last_peered":"2023-10-07T23:17:26.975957+0000","last_clean":"2023-10-07T23:17:26.975957+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:17:26.975957+0000","last_undegraded":"2023-10-07T23:17:26.975957+0000","last_fullsized":"2023-10-07T23:17:26.975957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006164+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:16:28.006164+0000","last_peered":"2023-10-07T23:16:28.006164+0000","last_clean":"2023-10-07T23:16:28.006164+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:16:28.006164+0000","last_undegraded":"2023-10-07T23:16:28.006164+0000","last_fullsized":"2023-10-07T23:16:28.006164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.000356+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:16:29.000356+0000","last_peered":"2023-10-07T23:16:29.000356+0000","last_clean":"2023-10-07T23:16:29.000356+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:16:29.000356+0000","last_undegraded":"2023-10-07T23:16:29.000356+0000","last_fullsized":"2023-10-07T23:16:29.000356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007922+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:16:28.007922+0000","last_peered":"2023-10-07T23:16:28.007922+0000","last_clean":"2023-10-07T23:16:28.007922+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:16:28.007922+0000","last_undegraded":"2023-10-07T23:16:28.007922+0000","last_fullsized":"2023-10-07T23:16:28.007922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017142+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:16:28.017142+0000","last_peered":"2023-10-07T23:16:28.017142+0000","last_clean":"2023-10-07T23:16:28.017142+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:16:28.017142+0000","last_undegraded":"2023-10-07T23:16:28.017142+0000","last_fullsized":"2023-10-07T23:16:28.017142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999614+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:16:27.999614+0000","last_peered":"2023-10-07T23:16:27.999614+0000","last_clean":"2023-10-07T23:16:27.999614+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:16:27.999614+0000","last_undegraded":"2023-10-07T23:16:27.999614+0000","last_fullsized":"2023-10-07T23:16:27.999614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005267+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:16:28.005267+0000","last_peered":"2023-10-07T23:16:28.005267+0000","last_clean":"2023-10-07T23:16:28.005267+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:16:28.005267+0000","last_undegraded":"2023-10-07T23:16:28.005267+0000","last_fullsized":"2023-10-07T23:16:28.005267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006062+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:16:28.006062+0000","last_peered":"2023-10-07T23:16:28.006062+0000","last_clean":"2023-10-07T23:16:28.006062+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:16:28.006062+0000","last_undegraded":"2023-10-07T23:16:28.006062+0000","last_fullsized":"2023-10-07T23:16:28.006062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.000876+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:16:29.000876+0000","last_peered":"2023-10-07T23:16:29.000876+0000","last_clean":"2023-10-07T23:16:29.000876+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:16:29.000876+0000","last_undegraded":"2023-10-07T23:16:29.000876+0000","last_fullsized":"2023-10-07T23:16:29.000876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'14","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975788+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:17:26.975788+0000","last_peered":"2023-10-07T23:17:26.975788+0000","last_clean":"2023-10-07T23:17:26.975788+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:17:26.975788+0000","last_undegraded":"2023-10-07T23:17:26.975788+0000","last_fullsized":"2023-10-07T23:17:26.975788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":98,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005882+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:16:29.005882+0000","last_peered":"2023-10-07T23:16:29.005882+0000","last_clean":"2023-10-07T23:16:29.005882+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:16:29.005882+0000","last_undegraded":"2023-10-07T23:16:29.005882+0000","last_fullsized":"2023-10-07T23:16:29.005882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005504+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:16:28.005504+0000","last_peered":"2023-10-07T23:16:28.005504+0000","last_clean":"2023-10-07T23:16:28.005504+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:16:28.005504+0000","last_undegraded":"2023-10-07T23:16:28.005504+0000","last_fullsized":"2023-10-07T23:16:28.005504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007745+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:16:28.007745+0000","last_peered":"2023-10-07T23:16:28.007745+0000","last_clean":"2023-10-07T23:16:28.007745+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:16:28.007745+0000","last_undegraded":"2023-10-07T23:16:28.007745+0000","last_fullsized":"2023-10-07T23:16:28.007745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999913+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:16:27.999913+0000","last_peered":"2023-10-07T23:16:27.999913+0000","last_clean":"2023-10-07T23:16:27.999913+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:16:27.999913+0000","last_undegraded":"2023-10-07T23:16:27.999913+0000","last_fullsized":"2023-10-07T23:16:27.999913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002941+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:16:28.002941+0000","last_peered":"2023-10-07T23:16:28.002941+0000","last_clean":"2023-10-07T23:16:28.002941+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:16:28.002941+0000","last_undegraded":"2023-10-07T23:16:28.002941+0000","last_fullsized":"2023-10-07T23:16:28.002941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005850+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:16:28.005850+0000","last_peered":"2023-10-07T23:16:28.005850+0000","last_clean":"2023-10-07T23:16:28.005850+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:16:28.005850+0000","last_undegraded":"2023-10-07T23:16:28.005850+0000","last_fullsized":"2023-10-07T23:16:28.005850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008325+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:16:28.008325+0000","last_peered":"2023-10-07T23:16:28.008325+0000","last_clean":"2023-10-07T23:16:28.008325+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:16:28.008325+0000","last_undegraded":"2023-10-07T23:16:28.008325+0000","last_fullsized":"2023-10-07T23:16:28.008325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045000+0000","last_change":"2023-10-07T23:16:29.044573+0000","last_active":"2023-10-07T23:16:29.045000+0000","last_peered":"2023-10-07T23:16:29.045000+0000","last_clean":"2023-10-07T23:16:29.045000+0000","last_became_active":"2023-10-07T23:16:29.044228+0000","last_became_peered":"2023-10-07T23:16:29.044228+0000","last_unstale":"2023-10-07T23:16:29.045000+0000","last_undegraded":"2023-10-07T23:16:29.045000+0000","last_fullsized":"2023-10-07T23:16:29.045000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:31.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975131+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:17:25.975131+0000","last_peered":"2023-10-07T23:17:25.975131+0000","last_clean":"2023-10-07T23:17:25.975131+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:17:25.975131+0000","last_undegraded":"2023-10-07T23:17:25.975131+0000","last_fullsized":"2023-10-07T23:17:25.975131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":798,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007188+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:16:28.007188+0000","last_peered":"2023-10-07T23:16:28.007188+0000","last_clean":"2023-10-07T23:16:28.007188+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:16:28.007188+0000","last_undegraded":"2023-10-07T23:16:28.007188+0000","last_fullsized":"2023-10-07T23:16:28.007188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003129+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:16:28.003129+0000","last_peered":"2023-10-07T23:16:28.003129+0000","last_clean":"2023-10-07T23:16:28.003129+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:16:28.003129+0000","last_undegraded":"2023-10-07T23:16:28.003129+0000","last_fullsized":"2023-10-07T23:16:28.003129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005464+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:16:29.005464+0000","last_peered":"2023-10-07T23:16:29.005464+0000","last_clean":"2023-10-07T23:16:29.005464+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:16:29.005464+0000","last_undegraded":"2023-10-07T23:16:29.005464+0000","last_fullsized":"2023-10-07T23:16:29.005464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003744+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:16:28.003744+0000","last_peered":"2023-10-07T23:16:28.003744+0000","last_clean":"2023-10-07T23:16:28.003744+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:16:28.003744+0000","last_undegraded":"2023-10-07T23:16:28.003744+0000","last_fullsized":"2023-10-07T23:16:28.003744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003691+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:16:28.003691+0000","last_peered":"2023-10-07T23:16:28.003691+0000","last_clean":"2023-10-07T23:16:28.003691+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:16:28.003691+0000","last_undegraded":"2023-10-07T23:16:28.003691+0000","last_fullsized":"2023-10-07T23:16:28.003691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005956+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:16:29.005956+0000","last_peered":"2023-10-07T23:16:29.005956+0000","last_clean":"2023-10-07T23:16:29.005956+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:16:29.005956+0000","last_undegraded":"2023-10-07T23:16:29.005956+0000","last_fullsized":"2023-10-07T23:16:29.005956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003423+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:16:28.003423+0000","last_peered":"2023-10-07T23:16:28.003423+0000","last_clean":"2023-10-07T23:16:28.003423+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:16:28.003423+0000","last_undegraded":"2023-10-07T23:16:28.003423+0000","last_fullsized":"2023-10-07T23:16:28.003423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005585+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:16:29.005585+0000","last_peered":"2023-10-07T23:16:29.005585+0000","last_clean":"2023-10-07T23:16:29.005585+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:16:29.005585+0000","last_undegraded":"2023-10-07T23:16:29.005585+0000","last_fullsized":"2023-10-07T23:16:29.005585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.001427+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:16:29.001427+0000","last_peered":"2023-10-07T23:16:29.001427+0000","last_clean":"2023-10-07T23:16:29.001427+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:16:29.001427+0000","last_undegraded":"2023-10-07T23:16:29.001427+0000","last_fullsized":"2023-10-07T23:16:29.001427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003225+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:16:28.003225+0000","last_peered":"2023-10-07T23:16:28.003225+0000","last_clean":"2023-10-07T23:16:28.003225+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:16:28.003225+0000","last_undegraded":"2023-10-07T23:16:28.003225+0000","last_fullsized":"2023-10-07T23:16:28.003225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001591+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:16:28.001591+0000","last_peered":"2023-10-07T23:16:28.001591+0000","last_clean":"2023-10-07T23:16:28.001591+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:16:28.001591+0000","last_undegraded":"2023-10-07T23:16:28.001591+0000","last_fullsized":"2023-10-07T23:16:28.001591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.986097+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:17:25.986097+0000","last_peered":"2023-10-07T23:17:25.986097+0000","last_clean":"2023-10-07T23:17:25.986097+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:17:25.986097+0000","last_undegraded":"2023-10-07T23:17:25.986097+0000","last_fullsized":"2023-10-07T23:17:25.986097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003060+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:16:28.003060+0000","last_peered":"2023-10-07T23:16:28.003060+0000","last_clean":"2023-10-07T23:16:28.003060+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:16:28.003060+0000","last_undegraded":"2023-10-07T23:16:28.003060+0000","last_fullsized":"2023-10-07T23:16:28.003060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999866+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:16:27.999866+0000","last_peered":"2023-10-07T23:16:27.999866+0000","last_clean":"2023-10-07T23:16:27.999866+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:16:27.999866+0000","last_undegraded":"2023-10-07T23:16:27.999866+0000","last_fullsized":"2023-10-07T23:16:27.999866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004005+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:16:28.004005+0000","last_peered":"2023-10-07T23:16:28.004005+0000","last_clean":"2023-10-07T23:16:28.004005+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:16:28.004005+0000","last_undegraded":"2023-10-07T23:16:28.004005+0000","last_fullsized":"2023-10-07T23:16:28.004005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975984+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:17:26.975984+0000","last_peered":"2023-10-07T23:17:26.975984+0000","last_clean":"2023-10-07T23:17:26.975984+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:17:26.975984+0000","last_undegraded":"2023-10-07T23:17:26.975984+0000","last_fullsized":"2023-10-07T23:17:26.975984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007777+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:16:28.007777+0000","last_peered":"2023-10-07T23:16:28.007777+0000","last_clean":"2023-10-07T23:16:28.007777+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:16:28.007777+0000","last_undegraded":"2023-10-07T23:16:28.007777+0000","last_fullsized":"2023-10-07T23:16:28.007777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008687+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:16:28.008687+0000","last_peered":"2023-10-07T23:16:28.008687+0000","last_clean":"2023-10-07T23:16:28.008687+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:16:28.008687+0000","last_undegraded":"2023-10-07T23:16:28.008687+0000","last_fullsized":"2023-10-07T23:16:28.008687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999386+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:16:27.999386+0000","last_peered":"2023-10-07T23:16:27.999386+0000","last_clean":"2023-10-07T23:16:27.999386+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:16:27.999386+0000","last_undegraded":"2023-10-07T23:16:27.999386+0000","last_fullsized":"2023-10-07T23:16:27.999386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007059+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:16:28.007059+0000","last_peered":"2023-10-07T23:16:28.007059+0000","last_clean":"2023-10-07T23:16:28.007059+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:16:28.007059+0000","last_undegraded":"2023-10-07T23:16:28.007059+0000","last_fullsized":"2023-10-07T23:16:28.007059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005716+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:16:28.005716+0000","last_peered":"2023-10-07T23:16:28.005716+0000","last_clean":"2023-10-07T23:16:28.005716+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:16:28.005716+0000","last_undegraded":"2023-10-07T23:16:28.005716+0000","last_fullsized":"2023-10-07T23:16:28.005716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999364+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:16:27.999364+0000","last_peered":"2023-10-07T23:16:27.999364+0000","last_clean":"2023-10-07T23:16:27.999364+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:16:27.999364+0000","last_undegraded":"2023-10-07T23:16:27.999364+0000","last_fullsized":"2023-10-07T23:16:27.999364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.015552+0000","last_change":"2023-10-07T23:16:29.015552+0000","last_active":"2023-10-07T23:16:29.015552+0000","last_peered":"2023-10-07T23:16:29.015552+0000","last_clean":"2023-10-07T23:16:29.015552+0000","last_became_active":"2023-10-07T23:16:29.010895+0000","last_became_peered":"2023-10-07T23:16:29.010895+0000","last_unstale":"2023-10-07T23:16:29.015552+0000","last_undegraded":"2023-10-07T23:16:29.015552+0000","last_fullsized":"2023-10-07T23:16:29.015552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:40.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.012361+0000","last_change":"2023-10-07T23:16:29.012361+0000","last_active":"2023-10-07T23:16:29.012361+0000","last_peered":"2023-10-07T23:16:29.012361+0000","last_clean":"2023-10-07T23:16:29.012361+0000","last_became_active":"2023-10-07T23:16:29.011918+0000","last_became_peered":"2023-10-07T23:16:29.011918+0000","last_unstale":"2023-10-07T23:16:29.012361+0000","last_undegraded":"2023-10-07T23:16:29.012361+0000","last_fullsized":"2023-10-07T23:16:29.012361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:32.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.006149+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:16:29.006149+0000","last_peered":"2023-10-07T23:16:29.006149+0000","last_clean":"2023-10-07T23:16:29.006149+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:16:29.006149+0000","last_undegraded":"2023-10-07T23:16:29.006149+0000","last_fullsized":"2023-10-07T23:16:29.006149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001743+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:16:28.001743+0000","last_peered":"2023-10-07T23:16:28.001743+0000","last_clean":"2023-10-07T23:16:28.001743+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.001743+0000","last_undegraded":"2023-10-07T23:16:28.001743+0000","last_fullsized":"2023-10-07T23:16:28.001743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016218+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:16:28.016218+0000","last_peered":"2023-10-07T23:16:28.016218+0000","last_clean":"2023-10-07T23:16:28.016218+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:16:28.016218+0000","last_undegraded":"2023-10-07T23:16:28.016218+0000","last_fullsized":"2023-10-07T23:16:28.016218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.019598+0000","last_change":"2023-10-07T23:16:29.018993+0000","last_active":"2023-10-07T23:16:29.019598+0000","last_peered":"2023-10-07T23:16:29.019598+0000","last_clean":"2023-10-07T23:16:29.019598+0000","last_became_active":"2023-10-07T23:16:29.008350+0000","last_became_peered":"2023-10-07T23:16:29.008350+0000","last_unstale":"2023-10-07T23:16:29.019598+0000","last_undegraded":"2023-10-07T23:16:29.019598+0000","last_fullsized":"2023-10-07T23:16:29.019598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:16.236418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016364+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:16:28.016364+0000","last_peered":"2023-10-07T23:16:28.016364+0000","last_clean":"2023-10-07T23:16:28.016364+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:16:28.016364+0000","last_undegraded":"2023-10-07T23:16:28.016364+0000","last_fullsized":"2023-10-07T23:16:28.016364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006054+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:16:28.006054+0000","last_peered":"2023-10-07T23:16:28.006054+0000","last_clean":"2023-10-07T23:16:28.006054+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:16:28.006054+0000","last_undegraded":"2023-10-07T23:16:28.006054+0000","last_fullsized":"2023-10-07T23:16:28.006054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999879+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:16:27.999879+0000","last_peered":"2023-10-07T23:16:27.999879+0000","last_clean":"2023-10-07T23:16:27.999879+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:16:27.999879+0000","last_undegraded":"2023-10-07T23:16:27.999879+0000","last_fullsized":"2023-10-07T23:16:27.999879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012811+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:16:28.012811+0000","last_peered":"2023-10-07T23:16:28.012811+0000","last_clean":"2023-10-07T23:16:28.012811+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:16:28.012811+0000","last_undegraded":"2023-10-07T23:16:28.012811+0000","last_fullsized":"2023-10-07T23:16:28.012811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999561+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:16:27.999561+0000","last_peered":"2023-10-07T23:16:27.999561+0000","last_clean":"2023-10-07T23:16:27.999561+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:16:27.999561+0000","last_undegraded":"2023-10-07T23:16:27.999561+0000","last_fullsized":"2023-10-07T23:16:27.999561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005469+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:16:28.005469+0000","last_peered":"2023-10-07T23:16:28.005469+0000","last_clean":"2023-10-07T23:16:28.005469+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:16:28.005469+0000","last_undegraded":"2023-10-07T23:16:28.005469+0000","last_fullsized":"2023-10-07T23:16:28.005469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005566+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:16:29.005566+0000","last_peered":"2023-10-07T23:16:29.005566+0000","last_clean":"2023-10-07T23:16:29.005566+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:16:29.005566+0000","last_undegraded":"2023-10-07T23:16:29.005566+0000","last_fullsized":"2023-10-07T23:16:29.005566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.001347+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:16:29.001347+0000","last_peered":"2023-10-07T23:16:29.001347+0000","last_clean":"2023-10-07T23:16:29.001347+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:16:29.001347+0000","last_undegraded":"2023-10-07T23:16:29.001347+0000","last_fullsized":"2023-10-07T23:16:29.001347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008316+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:16:28.008316+0000","last_peered":"2023-10-07T23:16:28.008316+0000","last_clean":"2023-10-07T23:16:28.008316+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:16:28.008316+0000","last_undegraded":"2023-10-07T23:16:28.008316+0000","last_fullsized":"2023-10-07T23:16:28.008316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002767+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:16:28.002767+0000","last_peered":"2023-10-07T23:16:28.002767+0000","last_clean":"2023-10-07T23:16:28.002767+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:16:28.002767+0000","last_undegraded":"2023-10-07T23:16:28.002767+0000","last_fullsized":"2023-10-07T23:16:28.002767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999422+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:16:27.999422+0000","last_peered":"2023-10-07T23:16:27.999422+0000","last_clean":"2023-10-07T23:16:27.999422+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:16:27.999422+0000","last_undegraded":"2023-10-07T23:16:27.999422+0000","last_fullsized":"2023-10-07T23:16:27.999422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006299+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:16:28.006299+0000","last_peered":"2023-10-07T23:16:28.006299+0000","last_clean":"2023-10-07T23:16:28.006299+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:16:28.006299+0000","last_undegraded":"2023-10-07T23:16:28.006299+0000","last_fullsized":"2023-10-07T23:16:28.006299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014270+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:16:28.014270+0000","last_peered":"2023-10-07T23:16:28.014270+0000","last_clean":"2023-10-07T23:16:28.014270+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:16:28.014270+0000","last_undegraded":"2023-10-07T23:16:28.014270+0000","last_fullsized":"2023-10-07T23:16:28.014270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999515+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:16:27.999515+0000","last_peered":"2023-10-07T23:16:27.999515+0000","last_clean":"2023-10-07T23:16:27.999515+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:16:27.999515+0000","last_undegraded":"2023-10-07T23:16:27.999515+0000","last_fullsized":"2023-10-07T23:16:27.999515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.049631+0000","last_change":"2023-10-07T23:16:29.049214+0000","last_active":"2023-10-07T23:16:29.049631+0000","last_peered":"2023-10-07T23:16:29.049631+0000","last_clean":"2023-10-07T23:16:29.049631+0000","last_became_active":"2023-10-07T23:16:29.044778+0000","last_became_peered":"2023-10-07T23:16:29.044778+0000","last_unstale":"2023-10-07T23:16:29.049631+0000","last_undegraded":"2023-10-07T23:16:29.049631+0000","last_fullsized":"2023-10-07T23:16:29.049631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:38.061993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007511+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:16:28.007511+0000","last_peered":"2023-10-07T23:16:28.007511+0000","last_clean":"2023-10-07T23:16:28.007511+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:16:28.007511+0000","last_undegraded":"2023-10-07T23:16:28.007511+0000","last_fullsized":"2023-10-07T23:16:28.007511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005590+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:16:28.005590+0000","last_peered":"2023-10-07T23:16:28.005590+0000","last_clean":"2023-10-07T23:16:28.005590+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:16:28.005590+0000","last_undegraded":"2023-10-07T23:16:28.005590+0000","last_fullsized":"2023-10-07T23:16:28.005590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:22.983295+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:17:22.983295+0000","last_peered":"2023-10-07T23:17:22.983295+0000","last_clean":"2023-10-07T23:17:22.983295+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:17:22.983295+0000","last_undegraded":"2023-10-07T23:17:22.983295+0000","last_fullsized":"2023-10-07T23:17:22.983295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007939+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:16:28.007939+0000","last_peered":"2023-10-07T23:16:28.007939+0000","last_clean":"2023-10-07T23:16:28.007939+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:16:28.007939+0000","last_undegraded":"2023-10-07T23:16:28.007939+0000","last_fullsized":"2023-10-07T23:16:28.007939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012043+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:16:28.012043+0000","last_peered":"2023-10-07T23:16:28.012043+0000","last_clean":"2023-10-07T23:16:28.012043+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:16:28.012043+0000","last_undegraded":"2023-10-07T23:16:28.012043+0000","last_fullsized":"2023-10-07T23:16:28.012043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005371+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:16:28.005371+0000","last_peered":"2023-10-07T23:16:28.005371+0000","last_clean":"2023-10-07T23:16:28.005371+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:16:28.005371+0000","last_undegraded":"2023-10-07T23:16:28.005371+0000","last_fullsized":"2023-10-07T23:16:28.005371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007323+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:16:28.007323+0000","last_peered":"2023-10-07T23:16:28.007323+0000","last_clean":"2023-10-07T23:16:28.007323+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:16:28.007323+0000","last_undegraded":"2023-10-07T23:16:28.007323+0000","last_fullsized":"2023-10-07T23:16:28.007323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003056+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:16:28.003056+0000","last_peered":"2023-10-07T23:16:28.003056+0000","last_clean":"2023-10-07T23:16:28.003056+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:16:28.003056+0000","last_undegraded":"2023-10-07T23:16:28.003056+0000","last_fullsized":"2023-10-07T23:16:28.003056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.051993+0000","last_change":"2023-10-07T23:16:29.051993+0000","last_active":"2023-10-07T23:16:29.051993+0000","last_peered":"2023-10-07T23:16:29.051993+0000","last_clean":"2023-10-07T23:16:29.051993+0000","last_became_active":"2023-10-07T23:16:29.046123+0000","last_became_peered":"2023-10-07T23:16:29.046123+0000","last_unstale":"2023-10-07T23:16:29.051993+0000","last_undegraded":"2023-10-07T23:16:29.051993+0000","last_fullsized":"2023-10-07T23:16:29.051993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:52.174959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008468+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:16:28.008468+0000","last_peered":"2023-10-07T23:16:28.008468+0000","last_clean":"2023-10-07T23:16:28.008468+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:16:28.008468+0000","last_undegraded":"2023-10-07T23:16:28.008468+0000","last_fullsized":"2023-10-07T23:16:28.008468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002713+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:16:28.002713+0000","last_peered":"2023-10-07T23:16:28.002713+0000","last_clean":"2023-10-07T23:16:28.002713+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:16:28.002713+0000","last_undegraded":"2023-10-07T23:16:28.002713+0000","last_fullsized":"2023-10-07T23:16:28.002713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006794+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:16:28.006794+0000","last_peered":"2023-10-07T23:16:28.006794+0000","last_clean":"2023-10-07T23:16:28.006794+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:16:28.006794+0000","last_undegraded":"2023-10-07T23:16:28.006794+0000","last_fullsized":"2023-10-07T23:16:28.006794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006214+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:16:28.006214+0000","last_peered":"2023-10-07T23:16:28.006214+0000","last_clean":"2023-10-07T23:16:28.006214+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:16:28.006214+0000","last_undegraded":"2023-10-07T23:16:28.006214+0000","last_fullsized":"2023-10-07T23:16:28.006214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012237+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:16:28.012237+0000","last_peered":"2023-10-07T23:16:28.012237+0000","last_clean":"2023-10-07T23:16:28.012237+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:16:28.012237+0000","last_undegraded":"2023-10-07T23:16:28.012237+0000","last_fullsized":"2023-10-07T23:16:28.012237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044932+0000","last_change":"2023-10-07T23:16:29.044932+0000","last_active":"2023-10-07T23:16:29.044932+0000","last_peered":"2023-10-07T23:16:29.044932+0000","last_clean":"2023-10-07T23:16:29.044932+0000","last_became_active":"2023-10-07T23:16:29.044454+0000","last_became_peered":"2023-10-07T23:16:29.044454+0000","last_unstale":"2023-10-07T23:16:29.044932+0000","last_undegraded":"2023-10-07T23:16:29.044932+0000","last_fullsized":"2023-10-07T23:16:29.044932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:55.975049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003230+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:16:28.003230+0000","last_peered":"2023-10-07T23:16:28.003230+0000","last_clean":"2023-10-07T23:16:28.003230+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:16:28.003230+0000","last_undegraded":"2023-10-07T23:16:28.003230+0000","last_fullsized":"2023-10-07T23:16:28.003230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008782+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:16:28.008782+0000","last_peered":"2023-10-07T23:16:28.008782+0000","last_clean":"2023-10-07T23:16:28.008782+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:16:28.008782+0000","last_undegraded":"2023-10-07T23:16:28.008782+0000","last_fullsized":"2023-10-07T23:16:28.008782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015307+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:16:28.015307+0000","last_peered":"2023-10-07T23:16:28.015307+0000","last_clean":"2023-10-07T23:16:28.015307+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:16:28.015307+0000","last_undegraded":"2023-10-07T23:16:28.015307+0000","last_fullsized":"2023-10-07T23:16:28.015307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003934+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:16:28.003934+0000","last_peered":"2023-10-07T23:16:28.003934+0000","last_clean":"2023-10-07T23:16:28.003934+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:16:28.003934+0000","last_undegraded":"2023-10-07T23:16:28.003934+0000","last_fullsized":"2023-10-07T23:16:28.003934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008400+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:16:28.008400+0000","last_peered":"2023-10-07T23:16:28.008400+0000","last_clean":"2023-10-07T23:16:28.008400+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:16:28.008400+0000","last_undegraded":"2023-10-07T23:16:28.008400+0000","last_fullsized":"2023-10-07T23:16:28.008400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005732+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:16:28.005732+0000","last_peered":"2023-10-07T23:16:28.005732+0000","last_clean":"2023-10-07T23:16:28.005732+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:16:28.005732+0000","last_undegraded":"2023-10-07T23:16:28.005732+0000","last_fullsized":"2023-10-07T23:16:28.005732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000267+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:16:28.000267+0000","last_peered":"2023-10-07T23:16:28.000267+0000","last_clean":"2023-10-07T23:16:28.000267+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:16:28.000267+0000","last_undegraded":"2023-10-07T23:16:28.000267+0000","last_fullsized":"2023-10-07T23:16:28.000267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003067+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:16:28.003067+0000","last_peered":"2023-10-07T23:16:28.003067+0000","last_clean":"2023-10-07T23:16:28.003067+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:16:28.003067+0000","last_undegraded":"2023-10-07T23:16:28.003067+0000","last_fullsized":"2023-10-07T23:16:28.003067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":964,"num_read_kb":503,"num_write":211,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383715,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":495361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.621},"min":{"1min":0.42999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.755,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.53300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90000000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83299999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.002,"15min":1.002},"last":0.42399999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89200000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0189999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.57399999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.323,"15min":1.323},"last":0.81699999999999995}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.751}]},{"osd":9,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78100000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":0.76000000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0940000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82899999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298908,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446244,"statfs":{"total":95995035648,"available":95688953856,"internally_reserved":0,"allocated":1658880,"data_stored":1089923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.874,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94399999999999995,"5min":1.159,"15min":1.159},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.5,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.141,"5min":1.214,"15min":1.214},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.026,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0149999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76400000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.84299999999999997,"5min":1.171,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.86899999999999999,"15min":0.92700000000000005},"min":{"1min":0.625,"5min":0.502,"15min":0.502},"max":{"1min":0.871,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97299999999999998,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.75},"min":{"1min":0.628,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95899999999999996,"5min":1.179,"15min":1.179},"last":0.78100000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808879,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72799999999999998,"15min":0.76000000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.68200000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.054,"15min":1.054},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68700000000000006,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.495}]},{"osd":2,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.85299999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.77300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.278,"15min":1.278},"last":0.81699999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.622},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80800000000000005,"15min":0.84299999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63700000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79300000000000004,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.345,"15min":1.345},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82199999999999995,"15min":0.86399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.421,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.85899999999999999,"15min":0.91300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0860000000000001,"5min":1.579,"15min":1.579},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.98099999999999998},"min":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.95399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97699999999999998,"15min":1.0289999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.5880000000000001,"5min":1.603,"15min":1.603},"last":0.76200000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.97399999999999998,"15min":1.0389999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2470000000000001,"5min":1.782,"15min":1.782},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.92600000000000005,"15min":0.98499999999999999},"min":{"1min":0.60299999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6850000000000001,"5min":1.956,"15min":1.956},"last":0.82899999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.97299999999999998,"15min":1.0129999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.647},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.94799999999999995,"15min":1.002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94399999999999995,"5min":1.663,"15min":1.663},"last":0.89100000000000001}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.77200000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":1044480,"data_stored":495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.85299999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.438}]},{"osd":3,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67800000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52000000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.59299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.022,"5min":1.597,"15min":1.597},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1,"15min":1},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.80500000000000005,"15min":0.79400000000000004},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4259999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.755}]},{"osd":10,"last update":"Sat Oct 7 23:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76100000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":1.008,"15min":1.008},"last":0.82099999999999995}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.84099999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298888,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446264,"statfs":{"total":95995035648,"available":95688974336,"internally_reserved":0,"allocated":1638400,"data_stored":1087813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42499999999999999,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.504}]},{"osd":2,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59299999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":1.51,"15min":1.51},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.92800000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0329999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.74199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94599999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58199999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99399999999999999,"5min":1.056,"15min":1.056},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.52500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.002,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.754},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":494596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.752},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.65800000000000003},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.653,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.439},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.95999999999999996}]},{"osd":5,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.873}]},{"osd":7,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.82099999999999995,"15min":0.78700000000000003},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":0.879},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.677,"5min":6.524,"15min":6.524},"last":0.86199999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.089,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88600000000000001,"15min":0.88900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":2.8780000000000001},"last":0.82299999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.61099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65800000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.91900000000000004}]},{"osd":10,"last update":"Sat Oct 7 23:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.111,"15min":1.1890000000000001},"min":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.911,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0640000000000001,"15min":1.1299999999999999},"min":{"1min":0.68999999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.294,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.90700000000000003}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.51800000000000002},"min":{"1min":0.47699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.60999999999999999,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.84299999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.47199999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.433},"min":{"1min":0.33800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65100000000000002,"15min":0.56200000000000006},"min":{"1min":0.498,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.52100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70799999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":1.004,"15min":1.004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89800000000000002,"5min":1.107,"15min":1.107},"last":0.624},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80500000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0069999999999999,"5min":1.131,"15min":1.131},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94699999999999995,"5min":1.075,"15min":1.075},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.752},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.89000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.57099999999999995,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":5.4249999999999998,"5min":5.4249999999999998,"15min":5.4249999999999998},"last":0.65100000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.73499999999999999,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5990000000000002,"5min":6.5990000000000002,"15min":6.5990000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.879,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.76300000000000001,"15min":0.66300000000000003},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.4669999999999996,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.51300000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.81399999999999995,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.88300000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.49399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.6180000000000003,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":5.0609999999999999,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1339999999999999,"5min":1.381,"15min":1.381},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0049999999999999,"15min":0.97999999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":5.5439999999999996,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.68400000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.0029999999999999,"15min":0.94399999999999995},"min":{"1min":0.79300000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":6.6539999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.93600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446836,"statfs":{"total":95995035648,"available":95689560064,"internally_reserved":0,"allocated":1052672,"data_stored":499485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.54100000000000004},"min":{"1min":0.48899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.29899999999999999},"max":{"1min":0.88200000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Oct 7 23:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84999999999999998,"5min":0.871,"15min":0.871},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.746},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.45200000000000001},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42799999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88800000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.57399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85699999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.628,"15min":0.60099999999999998},"min":{"1min":0.59099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56999999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.65100000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.96899999999999997,"5min":1.024,"15min":1.024},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.75700000000000001,"15min":0.71399999999999997},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.03,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.77300000000000002,"15min":0.73099999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.76300000000000001}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":1040384,"data_stored":494675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"min":{"1min":0.34499999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80500000000000005,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.0169999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70899999999999996,"15min":0.629},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92400000000000004,"5min":2.387,"15min":2.387},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0680000000000001,"5min":1.218,"15min":1.218},"last":0.57399999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71699999999999997,"15min":0.64000000000000001},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95699999999999996,"5min":1.278,"15min":1.278},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.68600000000000005},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.01,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.139,"5min":0.80300000000000005,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":6.3940000000000001},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67200000000000004},"min":{"1min":0.46899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.64600000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.89200000000000002,"15min":0.81799999999999995},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":6.4160000000000004,"5min":6.4160000000000004,"15min":6.4160000000000004},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.93400000000000005,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.002,"5min":0.998,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.96099999999999997,"15min":0.99399999999999999},"min":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0269999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.95299999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.82999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":495155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56699999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41299999999999998,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.59199999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.55000000000000004},"min":{"1min":0.51000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.88,"15min":0.88},"last":0.66500000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.498}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.56499999999999995},"min":{"1min":0.52100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.64900000000000002,"15min":0.56899999999999995},"min":{"1min":0.60099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66800000000000004,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":1.004,"15min":1.004},"last":0.57499999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83699999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80400000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.81200000000000006,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.755},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69899999999999995}]},{"osd":10,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.89400000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446256,"statfs":{"total":95995035648,"available":95688966144,"internally_reserved":0,"allocated":1646592,"data_stored":1086978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.6230000000000002,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79800000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.50600000000000001},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71499999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41699999999999998,"15min":0.36899999999999999},"min":{"1min":0.36399999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.63200000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.373},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.38600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":3.827,"15min":3.827},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.55200000000000005},"min":{"1min":0.46400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90800000000000003,"5min":3.903,"15min":3.903},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.54600000000000004},"min":{"1min":0.52200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60799999999999998,"15min":0.55800000000000005},"min":{"1min":0.50800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54500000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.55200000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.749}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.47899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.63400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.747,"15min":0.748},"min":{"1min":0.61899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":1,"15min":1},"last":0.59999999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.65700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.625,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:17:29.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:17:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:29 smithi123 ceph-mon[122926]: pgmap v351: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-07T23:17:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-10-07T23:17:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-07T23:17:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:29 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-10-07T23:17:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:29 smithi123 ceph-mon[122926]: osdmap e92: 12 total, 12 up, 12 in 2023-10-07T23:17:29.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:29 smithi123 ceph-mon[122926]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:29 smithi086 ceph-mon[122131]: pgmap v351: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-07T23:17:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-10-07T23:17:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-07T23:17:29.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:29 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2023-10-07T23:17:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:29 smithi086 ceph-mon[122131]: osdmap e92: 12 total, 12 up, 12 in 2023-10-07T23:17:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:29 smithi086 ceph-mon[122131]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.603+0000 7efd3919d700 1 -- 172.21.15.67:0/2829139405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 msgr2=0x7efd34070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.603+0000 7efd3919d700 1 --2- 172.21.15.67:0/2829139405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd34070030 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd240049c0 tx=0x7efd24009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 -- 172.21.15.67:0/2829139405 shutdown_connections 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 --2- 172.21.15.67:0/2829139405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd34104b80 0x7efd34107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 --2- 172.21.15.67:0/2829139405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd34070030 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 --2- 172.21.15.67:0/2829139405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd34070570 0x7efd340709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 -- 172.21.15.67:0/2829139405 >> 172.21.15.67:0/2829139405 conn(0x7efd340f8460 msgr2=0x7efd340fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 -- 172.21.15.67:0/2829139405 shutdown_connections 2023-10-07T23:17:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 -- 172.21.15.67:0/2829139405 wait complete. 2023-10-07T23:17:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 Processor -- start 2023-10-07T23:17:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.604+0000 7efd3919d700 1 -- start start 2023-10-07T23:17:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3919d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd34070570 0x7efd3410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3919d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd341100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3919d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd34104b80 0x7efd34117150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3919d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34117760 con 0x7efd34071bc0 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3919d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd341178d0 con 0x7efd34104b80 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3919d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34117a90 con 0x7efd34070570 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd341100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd341100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46220/0 (socket says 172.21.15.67:46220) 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 -- 172.21.15.67:0/1141349704 learned_addr learned my addr 172.21.15.67:0/1141349704 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd3899c700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd34104b80 0x7efd34117150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd33fff700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd34070570 0x7efd3410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 -- 172.21.15.67:0/1141349704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd34070570 msgr2=0x7efd3410fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd34070570 0x7efd3410fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 -- 172.21.15.67:0/1141349704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd34104b80 msgr2=0x7efd34117150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd34104b80 0x7efd34117150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.605+0000 7efd337fe700 1 -- 172.21.15.67:0/1141349704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24005040 con 0x7efd34071bc0 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.606+0000 7efd337fe700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd341100c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ebf0 tx=0x7efd2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.606+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28007cc0 con 0x7efd34071bc0 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.606+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2800f040 con 0x7efd34071bc0 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.606+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd34117d10 con 0x7efd34071bc0 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.606+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28009460 con 0x7efd34071bc0 2023-10-07T23:17:29.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.606+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd34118230 con 0x7efd34071bc0 2023-10-07T23:17:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.607+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd341032d0 con 0x7efd34071bc0 2023-10-07T23:17:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.609+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7efd2800c680 con 0x7efd34071bc0 2023-10-07T23:17:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.610+0000 7efd30ff9700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd1c09b7f0 0x7efd1c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.611+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9103+0+0 (secure 0 0 0) 0x7efd280c4e20 con 0x7efd34071bc0 2023-10-07T23:17:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.611+0000 7efd33fff700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd1c09b7f0 0x7efd1c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:29.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.611+0000 7efd33fff700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd1c09b7f0 0x7efd1c09dca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd24007f50 tx=0x7efd24009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:29.614 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.614+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7efd28090280 con 0x7efd34071bc0 2023-10-07T23:17:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.814+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 --> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd3402b410 con 0x7efd1c09b7f0 2023-10-07T23:17:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.827+0000 7efd30ff9700 1 -- 172.21.15.67:0/1141349704 <== mgr.14152 v2:172.21.15.67:6800/2299203628 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426837 (secure 0 0 0) 0x7efd3402b410 con 0x7efd1c09b7f0 2023-10-07T23:17:29.833 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd1c09b7f0 msgr2=0x7efd1c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd1c09b7f0 0x7efd1c09dca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd24007f50 tx=0x7efd24009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 msgr2=0x7efd341100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd341100c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ebf0 tx=0x7efd2800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 shutdown_connections 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd34104b80 0x7efd34117150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7efd1c09b7f0 0x7efd1c09dca0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd34070570 0x7efd3410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 --2- 172.21.15.67:0/1141349704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd34071bc0 0x7efd341100c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 >> 172.21.15.67:0/1141349704 conn(0x7efd340f8460 msgr2=0x7efd340fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 shutdown_connections 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:29.832+0000 7efd3919d700 1 -- 172.21.15.67:0/1141349704 wait complete. 2023-10-07T23:17:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-07T23:17:29.853 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-10-07T23:17:29.056253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600431,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1090,"num_read_kb":612,"num_write":392,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":3581412,"kb_used_data":13956,"kb_used_omap":0,"kb_used_meta":3567360,"kb_avail":1121360412,"statfs":{"total":1151940427776,"available":1148273061888,"internally_reserved":0,"allocated":14290944,"data_stored":7730295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3652976640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":51,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.105690"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004139+0000","last_change":"2023-10-07T23:15:27.207282+0000","last_active":"2023-10-07T23:16:28.004139+0000","last_peered":"2023-10-07T23:16:28.004139+0000","last_clean":"2023-10-07T23:16:28.004139+0000","last_became_active":"2023-10-07T23:15:27.207006+0000","last_became_peered":"2023-10-07T23:15:27.207006+0000","last_unstale":"2023-10-07T23:16:28.004139+0000","last_undegraded":"2023-10-07T23:16:28.004139+0000","last_fullsized":"2023-10-07T23:16:28.004139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:14:50.355998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012736+0000","last_change":"2023-10-07T23:15:27.204310+0000","last_active":"2023-10-07T23:16:28.012736+0000","last_peered":"2023-10-07T23:16:28.012736+0000","last_clean":"2023-10-07T23:16:28.012736+0000","last_became_active":"2023-10-07T23:15:27.203902+0000","last_became_peered":"2023-10-07T23:15:27.203902+0000","last_unstale":"2023-10-07T23:16:28.012736+0000","last_undegraded":"2023-10-07T23:16:28.012736+0000","last_fullsized":"2023-10-07T23:16:28.012736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:28.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012507+0000","last_change":"2023-10-07T23:15:27.211376+0000","last_active":"2023-10-07T23:16:28.012507+0000","last_peered":"2023-10-07T23:16:28.012507+0000","last_clean":"2023-10-07T23:16:28.012507+0000","last_became_active":"2023-10-07T23:15:27.210275+0000","last_became_peered":"2023-10-07T23:15:27.210275+0000","last_unstale":"2023-10-07T23:16:28.012507+0000","last_undegraded":"2023-10-07T23:16:28.012507+0000","last_fullsized":"2023-10-07T23:16:28.012507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.189713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999764+0000","last_change":"2023-10-07T23:15:27.204291+0000","last_active":"2023-10-07T23:16:27.999764+0000","last_peered":"2023-10-07T23:16:27.999764+0000","last_clean":"2023-10-07T23:16:27.999764+0000","last_became_active":"2023-10-07T23:15:27.204020+0000","last_became_peered":"2023-10-07T23:15:27.204020+0000","last_unstale":"2023-10-07T23:16:27.999764+0000","last_undegraded":"2023-10-07T23:16:27.999764+0000","last_fullsized":"2023-10-07T23:16:27.999764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:28.361383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.008822+0000","last_change":"2023-10-07T23:15:27.214830+0000","last_active":"2023-10-07T23:16:29.008822+0000","last_peered":"2023-10-07T23:16:29.008822+0000","last_clean":"2023-10-07T23:16:29.008822+0000","last_became_active":"2023-10-07T23:15:27.214496+0000","last_became_peered":"2023-10-07T23:15:27.214496+0000","last_unstale":"2023-10-07T23:16:29.008822+0000","last_undegraded":"2023-10-07T23:16:29.008822+0000","last_fullsized":"2023-10-07T23:16:29.008822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:23.081899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003946+0000","last_change":"2023-10-07T23:15:27.197136+0000","last_active":"2023-10-07T23:16:28.003946+0000","last_peered":"2023-10-07T23:16:28.003946+0000","last_clean":"2023-10-07T23:16:28.003946+0000","last_became_active":"2023-10-07T23:15:27.196626+0000","last_became_peered":"2023-10-07T23:15:27.196626+0000","last_unstale":"2023-10-07T23:16:28.003946+0000","last_undegraded":"2023-10-07T23:16:28.003946+0000","last_fullsized":"2023-10-07T23:16:28.003946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:00.355973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007260+0000","last_change":"2023-10-07T23:15:27.195235+0000","last_active":"2023-10-07T23:16:28.007260+0000","last_peered":"2023-10-07T23:16:28.007260+0000","last_clean":"2023-10-07T23:16:28.007260+0000","last_became_active":"2023-10-07T23:15:27.194935+0000","last_became_peered":"2023-10-07T23:15:27.194935+0000","last_unstale":"2023-10-07T23:16:28.007260+0000","last_undegraded":"2023-10-07T23:16:28.007260+0000","last_fullsized":"2023-10-07T23:16:28.007260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:23.003559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015359+0000","last_change":"2023-10-07T23:15:27.210183+0000","last_active":"2023-10-07T23:16:28.015359+0000","last_peered":"2023-10-07T23:16:28.015359+0000","last_clean":"2023-10-07T23:16:28.015359+0000","last_became_active":"2023-10-07T23:15:27.209864+0000","last_became_peered":"2023-10-07T23:15:27.209864+0000","last_unstale":"2023-10-07T23:16:28.015359+0000","last_undegraded":"2023-10-07T23:16:28.015359+0000","last_fullsized":"2023-10-07T23:16:28.015359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:27.639870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006337+0000","last_change":"2023-10-07T23:15:27.203835+0000","last_active":"2023-10-07T23:16:28.006337+0000","last_peered":"2023-10-07T23:16:28.006337+0000","last_clean":"2023-10-07T23:16:28.006337+0000","last_became_active":"2023-10-07T23:15:27.203144+0000","last_became_peered":"2023-10-07T23:15:27.203144+0000","last_unstale":"2023-10-07T23:16:28.006337+0000","last_undegraded":"2023-10-07T23:16:28.006337+0000","last_fullsized":"2023-10-07T23:16:28.006337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:47.061873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003351+0000","last_change":"2023-10-07T23:15:27.204309+0000","last_active":"2023-10-07T23:16:28.003351+0000","last_peered":"2023-10-07T23:16:28.003351+0000","last_clean":"2023-10-07T23:16:28.003351+0000","last_became_active":"2023-10-07T23:15:27.204136+0000","last_became_peered":"2023-10-07T23:15:27.204136+0000","last_unstale":"2023-10-07T23:16:28.003351+0000","last_undegraded":"2023-10-07T23:16:28.003351+0000","last_fullsized":"2023-10-07T23:16:28.003351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:13.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007883+0000","last_change":"2023-10-07T23:15:27.201121+0000","last_active":"2023-10-07T23:16:28.007883+0000","last_peered":"2023-10-07T23:16:28.007883+0000","last_clean":"2023-10-07T23:16:28.007883+0000","last_became_active":"2023-10-07T23:15:27.200848+0000","last_became_peered":"2023-10-07T23:15:27.200848+0000","last_unstale":"2023-10-07T23:16:28.007883+0000","last_undegraded":"2023-10-07T23:16:28.007883+0000","last_fullsized":"2023-10-07T23:16:28.007883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:23.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999630+0000","last_change":"2023-10-07T23:15:27.203716+0000","last_active":"2023-10-07T23:16:27.999630+0000","last_peered":"2023-10-07T23:16:27.999630+0000","last_clean":"2023-10-07T23:16:27.999630+0000","last_became_active":"2023-10-07T23:15:27.203408+0000","last_became_peered":"2023-10-07T23:15:27.203408+0000","last_unstale":"2023-10-07T23:16:27.999630+0000","last_undegraded":"2023-10-07T23:16:27.999630+0000","last_fullsized":"2023-10-07T23:16:27.999630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:51.468132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000842+0000","last_change":"2023-10-07T23:15:27.197707+0000","last_active":"2023-10-07T23:16:28.000842+0000","last_peered":"2023-10-07T23:16:28.000842+0000","last_clean":"2023-10-07T23:16:28.000842+0000","last_became_active":"2023-10-07T23:15:27.197375+0000","last_became_peered":"2023-10-07T23:15:27.197375+0000","last_unstale":"2023-10-07T23:16:28.000842+0000","last_undegraded":"2023-10-07T23:16:28.000842+0000","last_fullsized":"2023-10-07T23:16:28.000842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:27.569871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017901+0000","last_change":"2023-10-07T23:15:27.203968+0000","last_active":"2023-10-07T23:16:28.017901+0000","last_peered":"2023-10-07T23:16:28.017901+0000","last_clean":"2023-10-07T23:16:28.017901+0000","last_became_active":"2023-10-07T23:15:27.203593+0000","last_became_peered":"2023-10-07T23:15:27.203593+0000","last_unstale":"2023-10-07T23:16:28.017901+0000","last_undegraded":"2023-10-07T23:16:28.017901+0000","last_fullsized":"2023-10-07T23:16:28.017901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:55.622746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.011942+0000","last_change":"2023-10-07T23:15:27.208989+0000","last_active":"2023-10-07T23:16:28.011942+0000","last_peered":"2023-10-07T23:16:28.011942+0000","last_clean":"2023-10-07T23:16:28.011942+0000","last_became_active":"2023-10-07T23:15:27.207998+0000","last_became_peered":"2023-10-07T23:15:27.207998+0000","last_unstale":"2023-10-07T23:16:28.011942+0000","last_undegraded":"2023-10-07T23:16:28.011942+0000","last_fullsized":"2023-10-07T23:16:28.011942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:05.684842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005454+0000","last_change":"2023-10-07T23:15:27.202611+0000","last_active":"2023-10-07T23:16:28.005454+0000","last_peered":"2023-10-07T23:16:28.005454+0000","last_clean":"2023-10-07T23:16:28.005454+0000","last_became_active":"2023-10-07T23:15:27.202338+0000","last_became_peered":"2023-10-07T23:15:27.202338+0000","last_unstale":"2023-10-07T23:16:28.005454+0000","last_undegraded":"2023-10-07T23:16:28.005454+0000","last_fullsized":"2023-10-07T23:16:28.005454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:04.663790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004354+0000","last_change":"2023-10-07T23:15:27.207106+0000","last_active":"2023-10-07T23:16:28.004354+0000","last_peered":"2023-10-07T23:16:28.004354+0000","last_clean":"2023-10-07T23:16:28.004354+0000","last_became_active":"2023-10-07T23:15:27.206888+0000","last_became_peered":"2023-10-07T23:15:27.206888+0000","last_unstale":"2023-10-07T23:16:28.004354+0000","last_undegraded":"2023-10-07T23:16:28.004354+0000","last_fullsized":"2023-10-07T23:16:28.004354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:47.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.002524+0000","last_change":"2023-10-07T23:15:27.199185+0000","last_active":"2023-10-07T23:16:29.002524+0000","last_peered":"2023-10-07T23:16:29.002524+0000","last_clean":"2023-10-07T23:16:29.002524+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:29.002524+0000","last_undegraded":"2023-10-07T23:16:29.002524+0000","last_fullsized":"2023-10-07T23:16:29.002524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:50.193895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008489+0000","last_change":"2023-10-07T23:15:27.206234+0000","last_active":"2023-10-07T23:16:28.008489+0000","last_peered":"2023-10-07T23:16:28.008489+0000","last_clean":"2023-10-07T23:16:28.008489+0000","last_became_active":"2023-10-07T23:15:27.205983+0000","last_became_peered":"2023-10-07T23:15:27.205983+0000","last_unstale":"2023-10-07T23:16:28.008489+0000","last_undegraded":"2023-10-07T23:16:28.008489+0000","last_fullsized":"2023-10-07T23:16:28.008489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:23.127100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006630+0000","last_change":"2023-10-07T23:15:31.389209+0000","last_active":"2023-10-07T23:16:28.006630+0000","last_peered":"2023-10-07T23:16:28.006630+0000","last_clean":"2023-10-07T23:16:28.006630+0000","last_became_active":"2023-10-07T23:15:31.388876+0000","last_became_peered":"2023-10-07T23:15:31.388876+0000","last_unstale":"2023-10-07T23:16:28.006630+0000","last_undegraded":"2023-10-07T23:16:28.006630+0000","last_fullsized":"2023-10-07T23:16:28.006630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:41.637733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":125,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:27.987892+0000","last_change":"2023-10-07T23:15:26.358553+0000","last_active":"2023-10-07T23:17:27.987892+0000","last_peered":"2023-10-07T23:17:27.987892+0000","last_clean":"2023-10-07T23:17:27.987892+0000","last_became_active":"2023-10-07T23:15:26.358103+0000","last_became_peered":"2023-10-07T23:15:26.358103+0000","last_unstale":"2023-10-07T23:17:27.987892+0000","last_undegraded":"2023-10-07T23:17:27.987892+0000","last_fullsized":"2023-10-07T23:17:27.987892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:28.521650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014500+0000","last_change":"2023-10-07T23:15:27.217501+0000","last_active":"2023-10-07T23:16:28.014500+0000","last_peered":"2023-10-07T23:16:28.014500+0000","last_clean":"2023-10-07T23:16:28.014500+0000","last_became_active":"2023-10-07T23:15:27.217203+0000","last_became_peered":"2023-10-07T23:15:27.217203+0000","last_unstale":"2023-10-07T23:16:28.014500+0000","last_undegraded":"2023-10-07T23:16:28.014500+0000","last_fullsized":"2023-10-07T23:16:28.014500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:53:35.928727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999223+0000","last_change":"2023-10-07T23:15:31.386077+0000","last_active":"2023-10-07T23:16:27.999223+0000","last_peered":"2023-10-07T23:16:27.999223+0000","last_clean":"2023-10-07T23:16:27.999223+0000","last_became_active":"2023-10-07T23:15:31.385583+0000","last_became_peered":"2023-10-07T23:15:31.385583+0000","last_unstale":"2023-10-07T23:16:27.999223+0000","last_undegraded":"2023-10-07T23:16:27.999223+0000","last_fullsized":"2023-10-07T23:16:27.999223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:33.123988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014597+0000","last_change":"2023-10-07T23:15:26.362784+0000","last_active":"2023-10-07T23:16:28.014597+0000","last_peered":"2023-10-07T23:16:28.014597+0000","last_clean":"2023-10-07T23:16:28.014597+0000","last_became_active":"2023-10-07T23:15:26.362011+0000","last_became_peered":"2023-10-07T23:15:26.362011+0000","last_unstale":"2023-10-07T23:16:28.014597+0000","last_undegraded":"2023-10-07T23:16:28.014597+0000","last_fullsized":"2023-10-07T23:16:28.014597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:37.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012905+0000","last_change":"2023-10-07T23:15:27.211548+0000","last_active":"2023-10-07T23:16:28.012905+0000","last_peered":"2023-10-07T23:16:28.012905+0000","last_clean":"2023-10-07T23:16:28.012905+0000","last_became_active":"2023-10-07T23:15:27.209871+0000","last_became_peered":"2023-10-07T23:15:27.209871+0000","last_unstale":"2023-10-07T23:16:28.012905+0000","last_undegraded":"2023-10-07T23:16:28.012905+0000","last_fullsized":"2023-10-07T23:16:28.012905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:26.369285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999373+0000","last_change":"2023-10-07T23:15:31.389239+0000","last_active":"2023-10-07T23:16:27.999373+0000","last_peered":"2023-10-07T23:16:27.999373+0000","last_clean":"2023-10-07T23:16:27.999373+0000","last_became_active":"2023-10-07T23:15:31.388831+0000","last_became_peered":"2023-10-07T23:15:31.388831+0000","last_unstale":"2023-10-07T23:16:27.999373+0000","last_undegraded":"2023-10-07T23:16:27.999373+0000","last_fullsized":"2023-10-07T23:16:27.999373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:20.518208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.007402+0000","last_change":"2023-10-07T23:15:26.359235+0000","last_active":"2023-10-07T23:16:29.007402+0000","last_peered":"2023-10-07T23:16:29.007402+0000","last_clean":"2023-10-07T23:16:29.007402+0000","last_became_active":"2023-10-07T23:15:26.358945+0000","last_became_peered":"2023-10-07T23:15:26.358945+0000","last_unstale":"2023-10-07T23:16:29.007402+0000","last_undegraded":"2023-10-07T23:16:29.007402+0000","last_fullsized":"2023-10-07T23:16:29.007402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:08.686039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005917+0000","last_change":"2023-10-07T23:15:27.206115+0000","last_active":"2023-10-07T23:16:28.005917+0000","last_peered":"2023-10-07T23:16:28.005917+0000","last_clean":"2023-10-07T23:16:28.005917+0000","last_became_active":"2023-10-07T23:15:27.205751+0000","last_became_peered":"2023-10-07T23:15:27.205751+0000","last_unstale":"2023-10-07T23:16:28.005917+0000","last_undegraded":"2023-10-07T23:16:28.005917+0000","last_fullsized":"2023-10-07T23:16:28.005917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:25.721539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999220+0000","last_change":"2023-10-07T23:15:31.386027+0000","last_active":"2023-10-07T23:16:27.999220+0000","last_peered":"2023-10-07T23:16:27.999220+0000","last_clean":"2023-10-07T23:16:27.999220+0000","last_became_active":"2023-10-07T23:15:31.385418+0000","last_became_peered":"2023-10-07T23:15:31.385418+0000","last_unstale":"2023-10-07T23:16:27.999220+0000","last_undegraded":"2023-10-07T23:16:27.999220+0000","last_fullsized":"2023-10-07T23:16:27.999220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:07.940876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.006020+0000","last_change":"2023-10-07T23:15:26.352837+0000","last_active":"2023-10-07T23:16:29.006020+0000","last_peered":"2023-10-07T23:16:29.006020+0000","last_clean":"2023-10-07T23:16:29.006020+0000","last_became_active":"2023-10-07T23:15:26.352519+0000","last_became_peered":"2023-10-07T23:15:26.352519+0000","last_unstale":"2023-10-07T23:16:29.006020+0000","last_undegraded":"2023-10-07T23:16:29.006020+0000","last_fullsized":"2023-10-07T23:16:29.006020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:53.380409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"peering","last_fresh":"2023-10-07T23:17:28.058734+0000","last_change":"2023-10-07T23:17:28.058734+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:17:28.058734+0000","last_undegraded":"2023-10-07T23:17:28.058734+0000","last_fullsized":"2023-10-07T23:17:28.058734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:45:18.855595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008264+0000","last_change":"2023-10-07T23:15:31.378835+0000","last_active":"2023-10-07T23:16:28.008264+0000","last_peered":"2023-10-07T23:16:28.008264+0000","last_clean":"2023-10-07T23:16:28.008264+0000","last_became_active":"2023-10-07T23:15:31.378507+0000","last_became_peered":"2023-10-07T23:15:31.378507+0000","last_unstale":"2023-10-07T23:16:28.008264+0000","last_undegraded":"2023-10-07T23:16:28.008264+0000","last_fullsized":"2023-10-07T23:16:28.008264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.040906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975355+0000","last_change":"2023-10-07T23:15:26.359462+0000","last_active":"2023-10-07T23:17:25.975355+0000","last_peered":"2023-10-07T23:17:25.975355+0000","last_clean":"2023-10-07T23:17:25.975355+0000","last_became_active":"2023-10-07T23:15:26.359180+0000","last_became_peered":"2023-10-07T23:15:26.359180+0000","last_unstale":"2023-10-07T23:17:25.975355+0000","last_undegraded":"2023-10-07T23:17:25.975355+0000","last_fullsized":"2023-10-07T23:17:25.975355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:35.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005663+0000","last_change":"2023-10-07T23:15:27.199154+0000","last_active":"2023-10-07T23:16:29.005663+0000","last_peered":"2023-10-07T23:16:29.005663+0000","last_clean":"2023-10-07T23:16:29.005663+0000","last_became_active":"2023-10-07T23:15:27.195759+0000","last_became_peered":"2023-10-07T23:15:27.195759+0000","last_unstale":"2023-10-07T23:16:29.005663+0000","last_undegraded":"2023-10-07T23:16:29.005663+0000","last_fullsized":"2023-10-07T23:16:29.005663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:37.138337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.013091+0000","last_change":"2023-10-07T23:16:29.013091+0000","last_active":"2023-10-07T23:16:29.013091+0000","last_peered":"2023-10-07T23:16:29.013091+0000","last_clean":"2023-10-07T23:16:29.013091+0000","last_became_active":"2023-10-07T23:16:29.012680+0000","last_became_peered":"2023-10-07T23:16:29.012680+0000","last_unstale":"2023-10-07T23:16:29.013091+0000","last_undegraded":"2023-10-07T23:16:29.013091+0000","last_fullsized":"2023-10-07T23:16:29.013091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:13.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.050161+0000","last_change":"2023-10-07T23:16:29.050161+0000","last_active":"2023-10-07T23:16:29.050161+0000","last_peered":"2023-10-07T23:16:29.050161+0000","last_clean":"2023-10-07T23:16:29.050161+0000","last_became_active":"2023-10-07T23:16:29.044559+0000","last_became_peered":"2023-10-07T23:16:29.044559+0000","last_unstale":"2023-10-07T23:16:29.050161+0000","last_undegraded":"2023-10-07T23:16:29.050161+0000","last_fullsized":"2023-10-07T23:16:29.050161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:18.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006444+0000","last_change":"2023-10-07T23:15:27.206613+0000","last_active":"2023-10-07T23:16:28.006444+0000","last_peered":"2023-10-07T23:16:28.006444+0000","last_clean":"2023-10-07T23:16:28.006444+0000","last_became_active":"2023-10-07T23:15:27.206304+0000","last_became_peered":"2023-10-07T23:15:27.206304+0000","last_unstale":"2023-10-07T23:16:28.006444+0000","last_undegraded":"2023-10-07T23:16:28.006444+0000","last_fullsized":"2023-10-07T23:16:28.006444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:26.574520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045657+0000","last_change":"2023-10-07T23:16:29.045657+0000","last_active":"2023-10-07T23:16:29.045657+0000","last_peered":"2023-10-07T23:16:29.045657+0000","last_clean":"2023-10-07T23:16:29.045657+0000","last_became_active":"2023-10-07T23:16:29.045268+0000","last_became_peered":"2023-10-07T23:16:29.045268+0000","last_unstale":"2023-10-07T23:16:29.045657+0000","last_undegraded":"2023-10-07T23:16:29.045657+0000","last_fullsized":"2023-10-07T23:16:29.045657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:34.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999559+0000","last_change":"2023-10-07T23:15:26.351421+0000","last_active":"2023-10-07T23:16:27.999559+0000","last_peered":"2023-10-07T23:16:27.999559+0000","last_clean":"2023-10-07T23:16:27.999559+0000","last_became_active":"2023-10-07T23:15:26.350978+0000","last_became_peered":"2023-10-07T23:15:26.350978+0000","last_unstale":"2023-10-07T23:16:27.999559+0000","last_undegraded":"2023-10-07T23:16:27.999559+0000","last_fullsized":"2023-10-07T23:16:27.999559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:22.351586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016282+0000","last_change":"2023-10-07T23:15:27.210366+0000","last_active":"2023-10-07T23:16:28.016282+0000","last_peered":"2023-10-07T23:16:28.016282+0000","last_clean":"2023-10-07T23:16:28.016282+0000","last_became_active":"2023-10-07T23:15:27.209332+0000","last_became_peered":"2023-10-07T23:15:27.209332+0000","last_unstale":"2023-10-07T23:16:28.016282+0000","last_undegraded":"2023-10-07T23:16:28.016282+0000","last_fullsized":"2023-10-07T23:16:28.016282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.110298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012663+0000","last_change":"2023-10-07T23:15:31.395017+0000","last_active":"2023-10-07T23:16:28.012663+0000","last_peered":"2023-10-07T23:16:28.012663+0000","last_clean":"2023-10-07T23:16:28.012663+0000","last_became_active":"2023-10-07T23:15:31.394529+0000","last_became_peered":"2023-10-07T23:15:31.394529+0000","last_unstale":"2023-10-07T23:16:28.012663+0000","last_undegraded":"2023-10-07T23:16:28.012663+0000","last_fullsized":"2023-10-07T23:16:28.012663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:26:03.564642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003839+0000","last_change":"2023-10-07T23:15:26.354618+0000","last_active":"2023-10-07T23:16:28.003839+0000","last_peered":"2023-10-07T23:16:28.003839+0000","last_clean":"2023-10-07T23:16:28.003839+0000","last_became_active":"2023-10-07T23:15:26.354340+0000","last_became_peered":"2023-10-07T23:15:26.354340+0000","last_unstale":"2023-10-07T23:16:28.003839+0000","last_undegraded":"2023-10-07T23:16:28.003839+0000","last_fullsized":"2023-10-07T23:16:28.003839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:28:19.413133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.000644+0000","last_change":"2023-10-07T23:15:27.205387+0000","last_active":"2023-10-07T23:16:29.000644+0000","last_peered":"2023-10-07T23:16:29.000644+0000","last_clean":"2023-10-07T23:16:29.000644+0000","last_became_active":"2023-10-07T23:15:27.204970+0000","last_became_peered":"2023-10-07T23:15:27.204970+0000","last_unstale":"2023-10-07T23:16:29.000644+0000","last_undegraded":"2023-10-07T23:16:29.000644+0000","last_fullsized":"2023-10-07T23:16:29.000644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:49.482325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008218+0000","last_change":"2023-10-07T23:15:31.384258+0000","last_active":"2023-10-07T23:16:28.008218+0000","last_peered":"2023-10-07T23:16:28.008218+0000","last_clean":"2023-10-07T23:16:28.008218+0000","last_became_active":"2023-10-07T23:15:31.383897+0000","last_became_peered":"2023-10-07T23:15:31.383897+0000","last_unstale":"2023-10-07T23:16:28.008218+0000","last_undegraded":"2023-10-07T23:16:28.008218+0000","last_fullsized":"2023-10-07T23:16:28.008218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:30:32.299846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":207,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975957+0000","last_change":"2023-10-07T23:15:26.358279+0000","last_active":"2023-10-07T23:17:26.975957+0000","last_peered":"2023-10-07T23:17:26.975957+0000","last_clean":"2023-10-07T23:17:26.975957+0000","last_became_active":"2023-10-07T23:15:26.357970+0000","last_became_peered":"2023-10-07T23:15:26.357970+0000","last_unstale":"2023-10-07T23:17:26.975957+0000","last_undegraded":"2023-10-07T23:17:26.975957+0000","last_fullsized":"2023-10-07T23:17:26.975957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:33.154925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006164+0000","last_change":"2023-10-07T23:15:27.197825+0000","last_active":"2023-10-07T23:16:28.006164+0000","last_peered":"2023-10-07T23:16:28.006164+0000","last_clean":"2023-10-07T23:16:28.006164+0000","last_became_active":"2023-10-07T23:15:27.197478+0000","last_became_peered":"2023-10-07T23:15:27.197478+0000","last_unstale":"2023-10-07T23:16:28.006164+0000","last_undegraded":"2023-10-07T23:16:28.006164+0000","last_fullsized":"2023-10-07T23:16:28.006164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:17:03.898403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.000356+0000","last_change":"2023-10-07T23:15:31.391251+0000","last_active":"2023-10-07T23:16:29.000356+0000","last_peered":"2023-10-07T23:16:29.000356+0000","last_clean":"2023-10-07T23:16:29.000356+0000","last_became_active":"2023-10-07T23:15:31.390705+0000","last_became_peered":"2023-10-07T23:15:31.390705+0000","last_unstale":"2023-10-07T23:16:29.000356+0000","last_undegraded":"2023-10-07T23:16:29.000356+0000","last_fullsized":"2023-10-07T23:16:29.000356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:31.072885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007922+0000","last_change":"2023-10-07T23:15:26.358860+0000","last_active":"2023-10-07T23:16:28.007922+0000","last_peered":"2023-10-07T23:16:28.007922+0000","last_clean":"2023-10-07T23:16:28.007922+0000","last_became_active":"2023-10-07T23:15:26.358590+0000","last_became_peered":"2023-10-07T23:15:26.358590+0000","last_unstale":"2023-10-07T23:16:28.007922+0000","last_undegraded":"2023-10-07T23:16:28.007922+0000","last_fullsized":"2023-10-07T23:16:28.007922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:42.651045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.017142+0000","last_change":"2023-10-07T23:15:27.210188+0000","last_active":"2023-10-07T23:16:28.017142+0000","last_peered":"2023-10-07T23:16:28.017142+0000","last_clean":"2023-10-07T23:16:28.017142+0000","last_became_active":"2023-10-07T23:15:27.209288+0000","last_became_peered":"2023-10-07T23:15:27.209288+0000","last_unstale":"2023-10-07T23:16:28.017142+0000","last_undegraded":"2023-10-07T23:16:28.017142+0000","last_fullsized":"2023-10-07T23:16:28.017142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.270811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999614+0000","last_change":"2023-10-07T23:15:31.385544+0000","last_active":"2023-10-07T23:16:27.999614+0000","last_peered":"2023-10-07T23:16:27.999614+0000","last_clean":"2023-10-07T23:16:27.999614+0000","last_became_active":"2023-10-07T23:15:31.385028+0000","last_became_peered":"2023-10-07T23:15:31.385028+0000","last_unstale":"2023-10-07T23:16:27.999614+0000","last_undegraded":"2023-10-07T23:16:27.999614+0000","last_fullsized":"2023-10-07T23:16:27.999614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:47.407877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005267+0000","last_change":"2023-10-07T23:15:26.364385+0000","last_active":"2023-10-07T23:16:28.005267+0000","last_peered":"2023-10-07T23:16:28.005267+0000","last_clean":"2023-10-07T23:16:28.005267+0000","last_became_active":"2023-10-07T23:15:26.364158+0000","last_became_peered":"2023-10-07T23:15:26.364158+0000","last_unstale":"2023-10-07T23:16:28.005267+0000","last_undegraded":"2023-10-07T23:16:28.005267+0000","last_fullsized":"2023-10-07T23:16:28.005267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:43.889776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006062+0000","last_change":"2023-10-07T23:15:27.200541+0000","last_active":"2023-10-07T23:16:28.006062+0000","last_peered":"2023-10-07T23:16:28.006062+0000","last_clean":"2023-10-07T23:16:28.006062+0000","last_became_active":"2023-10-07T23:15:27.200240+0000","last_became_peered":"2023-10-07T23:15:27.200240+0000","last_unstale":"2023-10-07T23:16:28.006062+0000","last_undegraded":"2023-10-07T23:16:28.006062+0000","last_fullsized":"2023-10-07T23:16:28.006062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.037027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.000876+0000","last_change":"2023-10-07T23:15:31.383076+0000","last_active":"2023-10-07T23:16:29.000876+0000","last_peered":"2023-10-07T23:16:29.000876+0000","last_clean":"2023-10-07T23:16:29.000876+0000","last_became_active":"2023-10-07T23:15:31.382677+0000","last_became_peered":"2023-10-07T23:15:31.382677+0000","last_unstale":"2023-10-07T23:16:29.000876+0000","last_undegraded":"2023-10-07T23:16:29.000876+0000","last_fullsized":"2023-10-07T23:16:29.000876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:00:55.104734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'14","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975788+0000","last_change":"2023-10-07T23:15:26.359017+0000","last_active":"2023-10-07T23:17:26.975788+0000","last_peered":"2023-10-07T23:17:26.975788+0000","last_clean":"2023-10-07T23:17:26.975788+0000","last_became_active":"2023-10-07T23:15:26.358652+0000","last_became_peered":"2023-10-07T23:15:26.358652+0000","last_unstale":"2023-10-07T23:17:26.975788+0000","last_undegraded":"2023-10-07T23:17:26.975788+0000","last_fullsized":"2023-10-07T23:17:26.975788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:52:18.215882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":98,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005882+0000","last_change":"2023-10-07T23:15:27.202875+0000","last_active":"2023-10-07T23:16:29.005882+0000","last_peered":"2023-10-07T23:16:29.005882+0000","last_clean":"2023-10-07T23:16:29.005882+0000","last_became_active":"2023-10-07T23:15:27.202369+0000","last_became_peered":"2023-10-07T23:15:27.202369+0000","last_unstale":"2023-10-07T23:16:29.005882+0000","last_undegraded":"2023-10-07T23:16:29.005882+0000","last_fullsized":"2023-10-07T23:16:29.005882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:09.290746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005504+0000","last_change":"2023-10-07T23:15:31.385956+0000","last_active":"2023-10-07T23:16:28.005504+0000","last_peered":"2023-10-07T23:16:28.005504+0000","last_clean":"2023-10-07T23:16:28.005504+0000","last_became_active":"2023-10-07T23:15:31.385307+0000","last_became_peered":"2023-10-07T23:15:31.385307+0000","last_unstale":"2023-10-07T23:16:28.005504+0000","last_undegraded":"2023-10-07T23:16:28.005504+0000","last_fullsized":"2023-10-07T23:16:28.005504+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:25.483363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007745+0000","last_change":"2023-10-07T23:15:26.358364+0000","last_active":"2023-10-07T23:16:28.007745+0000","last_peered":"2023-10-07T23:16:28.007745+0000","last_clean":"2023-10-07T23:16:28.007745+0000","last_became_active":"2023-10-07T23:15:26.356972+0000","last_became_peered":"2023-10-07T23:15:26.356972+0000","last_unstale":"2023-10-07T23:16:28.007745+0000","last_undegraded":"2023-10-07T23:16:28.007745+0000","last_fullsized":"2023-10-07T23:16:28.007745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:09.922636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999913+0000","last_change":"2023-10-07T23:15:27.205275+0000","last_active":"2023-10-07T23:16:27.999913+0000","last_peered":"2023-10-07T23:16:27.999913+0000","last_clean":"2023-10-07T23:16:27.999913+0000","last_became_active":"2023-10-07T23:15:27.204922+0000","last_became_peered":"2023-10-07T23:15:27.204922+0000","last_unstale":"2023-10-07T23:16:27.999913+0000","last_undegraded":"2023-10-07T23:16:27.999913+0000","last_fullsized":"2023-10-07T23:16:27.999913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:32.346129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002941+0000","last_change":"2023-10-07T23:15:31.384950+0000","last_active":"2023-10-07T23:16:28.002941+0000","last_peered":"2023-10-07T23:16:28.002941+0000","last_clean":"2023-10-07T23:16:28.002941+0000","last_became_active":"2023-10-07T23:15:31.384549+0000","last_became_peered":"2023-10-07T23:15:31.384549+0000","last_unstale":"2023-10-07T23:16:28.002941+0000","last_undegraded":"2023-10-07T23:16:28.002941+0000","last_fullsized":"2023-10-07T23:16:28.002941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:21.440543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005850+0000","last_change":"2023-10-07T23:15:26.359244+0000","last_active":"2023-10-07T23:16:28.005850+0000","last_peered":"2023-10-07T23:16:28.005850+0000","last_clean":"2023-10-07T23:16:28.005850+0000","last_became_active":"2023-10-07T23:15:26.359002+0000","last_became_peered":"2023-10-07T23:15:26.359002+0000","last_unstale":"2023-10-07T23:16:28.005850+0000","last_undegraded":"2023-10-07T23:16:28.005850+0000","last_fullsized":"2023-10-07T23:16:28.005850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:26:12.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008325+0000","last_change":"2023-10-07T23:15:27.198346+0000","last_active":"2023-10-07T23:16:28.008325+0000","last_peered":"2023-10-07T23:16:28.008325+0000","last_clean":"2023-10-07T23:16:28.008325+0000","last_became_active":"2023-10-07T23:15:27.198066+0000","last_became_peered":"2023-10-07T23:15:27.198066+0000","last_unstale":"2023-10-07T23:16:28.008325+0000","last_undegraded":"2023-10-07T23:16:28.008325+0000","last_fullsized":"2023-10-07T23:16:28.008325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:35.103005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.045000+0000","last_change":"2023-10-07T23:16:29.044573+0000","last_active":"2023-10-07T23:16:29.045000+0000","last_peered":"2023-10-07T23:16:29.045000+0000","last_clean":"2023-10-07T23:16:29.045000+0000","last_became_active":"2023-10-07T23:16:29.044228+0000","last_became_peered":"2023-10-07T23:16:29.044228+0000","last_unstale":"2023-10-07T23:16:29.045000+0000","last_undegraded":"2023-10-07T23:16:29.045000+0000","last_fullsized":"2023-10-07T23:16:29.045000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:31.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.975131+0000","last_change":"2023-10-07T23:15:26.347046+0000","last_active":"2023-10-07T23:17:25.975131+0000","last_peered":"2023-10-07T23:17:25.975131+0000","last_clean":"2023-10-07T23:17:25.975131+0000","last_became_active":"2023-10-07T23:15:26.346187+0000","last_became_peered":"2023-10-07T23:15:26.346187+0000","last_unstale":"2023-10-07T23:17:25.975131+0000","last_undegraded":"2023-10-07T23:17:25.975131+0000","last_fullsized":"2023-10-07T23:17:25.975131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:11.132832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":798,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007188+0000","last_change":"2023-10-07T23:15:27.193288+0000","last_active":"2023-10-07T23:16:28.007188+0000","last_peered":"2023-10-07T23:16:28.007188+0000","last_clean":"2023-10-07T23:16:28.007188+0000","last_became_active":"2023-10-07T23:15:27.192945+0000","last_became_peered":"2023-10-07T23:15:27.192945+0000","last_unstale":"2023-10-07T23:16:28.007188+0000","last_undegraded":"2023-10-07T23:16:28.007188+0000","last_fullsized":"2023-10-07T23:16:28.007188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:28.993305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003129+0000","last_change":"2023-10-07T23:15:31.392770+0000","last_active":"2023-10-07T23:16:28.003129+0000","last_peered":"2023-10-07T23:16:28.003129+0000","last_clean":"2023-10-07T23:16:28.003129+0000","last_became_active":"2023-10-07T23:15:31.392381+0000","last_became_peered":"2023-10-07T23:15:31.392381+0000","last_unstale":"2023-10-07T23:16:28.003129+0000","last_undegraded":"2023-10-07T23:16:28.003129+0000","last_fullsized":"2023-10-07T23:16:28.003129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:19.183764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005464+0000","last_change":"2023-10-07T23:15:26.362895+0000","last_active":"2023-10-07T23:16:29.005464+0000","last_peered":"2023-10-07T23:16:29.005464+0000","last_clean":"2023-10-07T23:16:29.005464+0000","last_became_active":"2023-10-07T23:15:26.362624+0000","last_became_peered":"2023-10-07T23:15:26.362624+0000","last_unstale":"2023-10-07T23:16:29.005464+0000","last_undegraded":"2023-10-07T23:16:29.005464+0000","last_fullsized":"2023-10-07T23:16:29.005464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:55.726515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003744+0000","last_change":"2023-10-07T23:15:27.198039+0000","last_active":"2023-10-07T23:16:28.003744+0000","last_peered":"2023-10-07T23:16:28.003744+0000","last_clean":"2023-10-07T23:16:28.003744+0000","last_became_active":"2023-10-07T23:15:27.197351+0000","last_became_peered":"2023-10-07T23:15:27.197351+0000","last_unstale":"2023-10-07T23:16:28.003744+0000","last_undegraded":"2023-10-07T23:16:28.003744+0000","last_fullsized":"2023-10-07T23:16:28.003744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:23.075164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003691+0000","last_change":"2023-10-07T23:15:31.389034+0000","last_active":"2023-10-07T23:16:28.003691+0000","last_peered":"2023-10-07T23:16:28.003691+0000","last_clean":"2023-10-07T23:16:28.003691+0000","last_became_active":"2023-10-07T23:15:31.388610+0000","last_became_peered":"2023-10-07T23:15:31.388610+0000","last_unstale":"2023-10-07T23:16:28.003691+0000","last_undegraded":"2023-10-07T23:16:28.003691+0000","last_fullsized":"2023-10-07T23:16:28.003691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:50:46.277341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005956+0000","last_change":"2023-10-07T23:15:26.356816+0000","last_active":"2023-10-07T23:16:29.005956+0000","last_peered":"2023-10-07T23:16:29.005956+0000","last_clean":"2023-10-07T23:16:29.005956+0000","last_became_active":"2023-10-07T23:15:26.356392+0000","last_became_peered":"2023-10-07T23:15:26.356392+0000","last_unstale":"2023-10-07T23:16:29.005956+0000","last_undegraded":"2023-10-07T23:16:29.005956+0000","last_fullsized":"2023-10-07T23:16:29.005956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:27.350566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003423+0000","last_change":"2023-10-07T23:15:27.197989+0000","last_active":"2023-10-07T23:16:28.003423+0000","last_peered":"2023-10-07T23:16:28.003423+0000","last_clean":"2023-10-07T23:16:28.003423+0000","last_became_active":"2023-10-07T23:15:27.197695+0000","last_became_peered":"2023-10-07T23:15:27.197695+0000","last_unstale":"2023-10-07T23:16:28.003423+0000","last_undegraded":"2023-10-07T23:16:28.003423+0000","last_fullsized":"2023-10-07T23:16:28.003423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:16:21.813120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005585+0000","last_change":"2023-10-07T23:15:31.622223+0000","last_active":"2023-10-07T23:16:29.005585+0000","last_peered":"2023-10-07T23:16:29.005585+0000","last_clean":"2023-10-07T23:16:29.005585+0000","last_became_active":"2023-10-07T23:15:31.621700+0000","last_became_peered":"2023-10-07T23:15:31.621700+0000","last_unstale":"2023-10-07T23:16:29.005585+0000","last_undegraded":"2023-10-07T23:16:29.005585+0000","last_fullsized":"2023-10-07T23:16:29.005585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:41:37.089146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.001427+0000","last_change":"2023-10-07T23:15:26.364175+0000","last_active":"2023-10-07T23:16:29.001427+0000","last_peered":"2023-10-07T23:16:29.001427+0000","last_clean":"2023-10-07T23:16:29.001427+0000","last_became_active":"2023-10-07T23:15:26.363823+0000","last_became_peered":"2023-10-07T23:15:26.363823+0000","last_unstale":"2023-10-07T23:16:29.001427+0000","last_undegraded":"2023-10-07T23:16:29.001427+0000","last_fullsized":"2023-10-07T23:16:29.001427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:10.195998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003225+0000","last_change":"2023-10-07T23:15:27.203644+0000","last_active":"2023-10-07T23:16:28.003225+0000","last_peered":"2023-10-07T23:16:28.003225+0000","last_clean":"2023-10-07T23:16:28.003225+0000","last_became_active":"2023-10-07T23:15:27.203343+0000","last_became_peered":"2023-10-07T23:15:27.203343+0000","last_unstale":"2023-10-07T23:16:28.003225+0000","last_undegraded":"2023-10-07T23:16:28.003225+0000","last_fullsized":"2023-10-07T23:16:28.003225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:09:04.310202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001591+0000","last_change":"2023-10-07T23:15:31.393848+0000","last_active":"2023-10-07T23:16:28.001591+0000","last_peered":"2023-10-07T23:16:28.001591+0000","last_clean":"2023-10-07T23:16:28.001591+0000","last_became_active":"2023-10-07T23:15:31.392779+0000","last_became_peered":"2023-10-07T23:15:31.392779+0000","last_unstale":"2023-10-07T23:16:28.001591+0000","last_undegraded":"2023-10-07T23:16:28.001591+0000","last_fullsized":"2023-10-07T23:16:28.001591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:47:49.076295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:25.986097+0000","last_change":"2023-10-07T23:15:26.349251+0000","last_active":"2023-10-07T23:17:25.986097+0000","last_peered":"2023-10-07T23:17:25.986097+0000","last_clean":"2023-10-07T23:17:25.986097+0000","last_became_active":"2023-10-07T23:15:26.348921+0000","last_became_peered":"2023-10-07T23:15:26.348921+0000","last_unstale":"2023-10-07T23:17:25.986097+0000","last_undegraded":"2023-10-07T23:17:25.986097+0000","last_fullsized":"2023-10-07T23:17:25.986097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:19:40.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003060+0000","last_change":"2023-10-07T23:15:27.206719+0000","last_active":"2023-10-07T23:16:28.003060+0000","last_peered":"2023-10-07T23:16:28.003060+0000","last_clean":"2023-10-07T23:16:28.003060+0000","last_became_active":"2023-10-07T23:15:27.206420+0000","last_became_peered":"2023-10-07T23:15:27.206420+0000","last_unstale":"2023-10-07T23:16:28.003060+0000","last_undegraded":"2023-10-07T23:16:28.003060+0000","last_fullsized":"2023-10-07T23:16:28.003060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:29:11.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999866+0000","last_change":"2023-10-07T23:15:31.622055+0000","last_active":"2023-10-07T23:16:27.999866+0000","last_peered":"2023-10-07T23:16:27.999866+0000","last_clean":"2023-10-07T23:16:27.999866+0000","last_became_active":"2023-10-07T23:15:31.621615+0000","last_became_peered":"2023-10-07T23:15:31.621615+0000","last_unstale":"2023-10-07T23:16:27.999866+0000","last_undegraded":"2023-10-07T23:16:27.999866+0000","last_fullsized":"2023-10-07T23:16:27.999866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:07.501682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.004005+0000","last_change":"2023-10-07T23:15:27.200236+0000","last_active":"2023-10-07T23:16:28.004005+0000","last_peered":"2023-10-07T23:16:28.004005+0000","last_clean":"2023-10-07T23:16:28.004005+0000","last_became_active":"2023-10-07T23:15:27.199935+0000","last_became_peered":"2023-10-07T23:15:27.199935+0000","last_unstale":"2023-10-07T23:16:28.004005+0000","last_undegraded":"2023-10-07T23:16:28.004005+0000","last_fullsized":"2023-10-07T23:16:28.004005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:34:22.540488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:26.975984+0000","last_change":"2023-10-07T23:15:26.356635+0000","last_active":"2023-10-07T23:17:26.975984+0000","last_peered":"2023-10-07T23:17:26.975984+0000","last_clean":"2023-10-07T23:17:26.975984+0000","last_became_active":"2023-10-07T23:15:26.356332+0000","last_became_peered":"2023-10-07T23:15:26.356332+0000","last_unstale":"2023-10-07T23:17:26.975984+0000","last_undegraded":"2023-10-07T23:17:26.975984+0000","last_fullsized":"2023-10-07T23:17:26.975984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:38.278224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007777+0000","last_change":"2023-10-07T23:15:31.385459+0000","last_active":"2023-10-07T23:16:28.007777+0000","last_peered":"2023-10-07T23:16:28.007777+0000","last_clean":"2023-10-07T23:16:28.007777+0000","last_became_active":"2023-10-07T23:15:31.384715+0000","last_became_peered":"2023-10-07T23:15:31.384715+0000","last_unstale":"2023-10-07T23:16:28.007777+0000","last_undegraded":"2023-10-07T23:16:28.007777+0000","last_fullsized":"2023-10-07T23:16:28.007777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:35:30.755804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008687+0000","last_change":"2023-10-07T23:15:27.202998+0000","last_active":"2023-10-07T23:16:28.008687+0000","last_peered":"2023-10-07T23:16:28.008687+0000","last_clean":"2023-10-07T23:16:28.008687+0000","last_became_active":"2023-10-07T23:15:27.202666+0000","last_became_peered":"2023-10-07T23:15:27.202666+0000","last_unstale":"2023-10-07T23:16:28.008687+0000","last_undegraded":"2023-10-07T23:16:28.008687+0000","last_fullsized":"2023-10-07T23:16:28.008687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:34.181535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999386+0000","last_change":"2023-10-07T23:15:26.351932+0000","last_active":"2023-10-07T23:16:27.999386+0000","last_peered":"2023-10-07T23:16:27.999386+0000","last_clean":"2023-10-07T23:16:27.999386+0000","last_became_active":"2023-10-07T23:15:26.351694+0000","last_became_peered":"2023-10-07T23:15:26.351694+0000","last_unstale":"2023-10-07T23:16:27.999386+0000","last_undegraded":"2023-10-07T23:16:27.999386+0000","last_fullsized":"2023-10-07T23:16:27.999386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:03:43.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007059+0000","last_change":"2023-10-07T23:14:16.877205+0000","last_active":"2023-10-07T23:16:28.007059+0000","last_peered":"2023-10-07T23:16:28.007059+0000","last_clean":"2023-10-07T23:16:28.007059+0000","last_became_active":"2023-10-07T23:14:16.876656+0000","last_became_peered":"2023-10-07T23:14:16.876656+0000","last_unstale":"2023-10-07T23:16:28.007059+0000","last_undegraded":"2023-10-07T23:16:28.007059+0000","last_fullsized":"2023-10-07T23:16:28.007059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:10:50.793894+0000","last_clean_scrub_stamp":"2023-10-07T23:10:50.793894+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:57:39.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005716+0000","last_change":"2023-10-07T23:15:31.622447+0000","last_active":"2023-10-07T23:16:28.005716+0000","last_peered":"2023-10-07T23:16:28.005716+0000","last_clean":"2023-10-07T23:16:28.005716+0000","last_became_active":"2023-10-07T23:15:31.622075+0000","last_became_peered":"2023-10-07T23:15:31.622075+0000","last_unstale":"2023-10-07T23:16:28.005716+0000","last_undegraded":"2023-10-07T23:16:28.005716+0000","last_fullsized":"2023-10-07T23:16:28.005716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:42.513184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999364+0000","last_change":"2023-10-07T23:15:27.195221+0000","last_active":"2023-10-07T23:16:27.999364+0000","last_peered":"2023-10-07T23:16:27.999364+0000","last_clean":"2023-10-07T23:16:27.999364+0000","last_became_active":"2023-10-07T23:15:27.194859+0000","last_became_peered":"2023-10-07T23:15:27.194859+0000","last_unstale":"2023-10-07T23:16:27.999364+0000","last_undegraded":"2023-10-07T23:16:27.999364+0000","last_fullsized":"2023-10-07T23:16:27.999364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:45.800589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.015552+0000","last_change":"2023-10-07T23:16:29.015552+0000","last_active":"2023-10-07T23:16:29.015552+0000","last_peered":"2023-10-07T23:16:29.015552+0000","last_clean":"2023-10-07T23:16:29.015552+0000","last_became_active":"2023-10-07T23:16:29.010895+0000","last_became_peered":"2023-10-07T23:16:29.010895+0000","last_unstale":"2023-10-07T23:16:29.015552+0000","last_undegraded":"2023-10-07T23:16:29.015552+0000","last_fullsized":"2023-10-07T23:16:29.015552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:40.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.012361+0000","last_change":"2023-10-07T23:16:29.012361+0000","last_active":"2023-10-07T23:16:29.012361+0000","last_peered":"2023-10-07T23:16:29.012361+0000","last_clean":"2023-10-07T23:16:29.012361+0000","last_became_active":"2023-10-07T23:16:29.011918+0000","last_became_peered":"2023-10-07T23:16:29.011918+0000","last_unstale":"2023-10-07T23:16:29.012361+0000","last_undegraded":"2023-10-07T23:16:29.012361+0000","last_fullsized":"2023-10-07T23:16:29.012361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:32.803805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.006149+0000","last_change":"2023-10-07T23:15:26.362338+0000","last_active":"2023-10-07T23:16:29.006149+0000","last_peered":"2023-10-07T23:16:29.006149+0000","last_clean":"2023-10-07T23:16:29.006149+0000","last_became_active":"2023-10-07T23:15:26.362023+0000","last_became_peered":"2023-10-07T23:15:26.362023+0000","last_unstale":"2023-10-07T23:16:29.006149+0000","last_undegraded":"2023-10-07T23:16:29.006149+0000","last_fullsized":"2023-10-07T23:16:29.006149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:05:23.800071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.001743+0000","last_change":"2023-10-07T23:15:27.199205+0000","last_active":"2023-10-07T23:16:28.001743+0000","last_peered":"2023-10-07T23:16:28.001743+0000","last_clean":"2023-10-07T23:16:28.001743+0000","last_became_active":"2023-10-07T23:15:27.198881+0000","last_became_peered":"2023-10-07T23:15:27.198881+0000","last_unstale":"2023-10-07T23:16:28.001743+0000","last_undegraded":"2023-10-07T23:16:28.001743+0000","last_fullsized":"2023-10-07T23:16:28.001743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:34:36.525107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016218+0000","last_change":"2023-10-07T23:15:31.622021+0000","last_active":"2023-10-07T23:16:28.016218+0000","last_peered":"2023-10-07T23:16:28.016218+0000","last_clean":"2023-10-07T23:16:28.016218+0000","last_became_active":"2023-10-07T23:15:31.621554+0000","last_became_peered":"2023-10-07T23:15:31.621554+0000","last_unstale":"2023-10-07T23:16:28.016218+0000","last_undegraded":"2023-10-07T23:16:28.016218+0000","last_fullsized":"2023-10-07T23:16:28.016218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:49:29.788439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.019598+0000","last_change":"2023-10-07T23:16:29.018993+0000","last_active":"2023-10-07T23:16:29.019598+0000","last_peered":"2023-10-07T23:16:29.019598+0000","last_clean":"2023-10-07T23:16:29.019598+0000","last_became_active":"2023-10-07T23:16:29.008350+0000","last_became_peered":"2023-10-07T23:16:29.008350+0000","last_unstale":"2023-10-07T23:16:29.019598+0000","last_undegraded":"2023-10-07T23:16:29.019598+0000","last_fullsized":"2023-10-07T23:16:29.019598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:16.236418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.016364+0000","last_change":"2023-10-07T23:15:27.207025+0000","last_active":"2023-10-07T23:16:28.016364+0000","last_peered":"2023-10-07T23:16:28.016364+0000","last_clean":"2023-10-07T23:16:28.016364+0000","last_became_active":"2023-10-07T23:15:27.206680+0000","last_became_peered":"2023-10-07T23:15:27.206680+0000","last_unstale":"2023-10-07T23:16:28.016364+0000","last_undegraded":"2023-10-07T23:16:28.016364+0000","last_fullsized":"2023-10-07T23:16:28.016364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:01.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006054+0000","last_change":"2023-10-07T23:15:31.379391+0000","last_active":"2023-10-07T23:16:28.006054+0000","last_peered":"2023-10-07T23:16:28.006054+0000","last_clean":"2023-10-07T23:16:28.006054+0000","last_became_active":"2023-10-07T23:15:31.378558+0000","last_became_peered":"2023-10-07T23:15:31.378558+0000","last_unstale":"2023-10-07T23:16:28.006054+0000","last_undegraded":"2023-10-07T23:16:28.006054+0000","last_fullsized":"2023-10-07T23:16:28.006054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:09.984526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999879+0000","last_change":"2023-10-07T23:15:26.352106+0000","last_active":"2023-10-07T23:16:27.999879+0000","last_peered":"2023-10-07T23:16:27.999879+0000","last_clean":"2023-10-07T23:16:27.999879+0000","last_became_active":"2023-10-07T23:15:26.351797+0000","last_became_peered":"2023-10-07T23:15:26.351797+0000","last_unstale":"2023-10-07T23:16:27.999879+0000","last_undegraded":"2023-10-07T23:16:27.999879+0000","last_fullsized":"2023-10-07T23:16:27.999879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:12:27.170635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012811+0000","last_change":"2023-10-07T23:15:27.219784+0000","last_active":"2023-10-07T23:16:28.012811+0000","last_peered":"2023-10-07T23:16:28.012811+0000","last_clean":"2023-10-07T23:16:28.012811+0000","last_became_active":"2023-10-07T23:15:27.219283+0000","last_became_peered":"2023-10-07T23:15:27.219283+0000","last_unstale":"2023-10-07T23:16:28.012811+0000","last_undegraded":"2023-10-07T23:16:28.012811+0000","last_fullsized":"2023-10-07T23:16:28.012811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:55:53.399309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999561+0000","last_change":"2023-10-07T23:15:31.622621+0000","last_active":"2023-10-07T23:16:27.999561+0000","last_peered":"2023-10-07T23:16:27.999561+0000","last_clean":"2023-10-07T23:16:27.999561+0000","last_became_active":"2023-10-07T23:15:31.622166+0000","last_became_peered":"2023-10-07T23:15:31.622166+0000","last_unstale":"2023-10-07T23:16:27.999561+0000","last_undegraded":"2023-10-07T23:16:27.999561+0000","last_fullsized":"2023-10-07T23:16:27.999561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:05:12.545956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005469+0000","last_change":"2023-10-07T23:15:26.361065+0000","last_active":"2023-10-07T23:16:28.005469+0000","last_peered":"2023-10-07T23:16:28.005469+0000","last_clean":"2023-10-07T23:16:28.005469+0000","last_became_active":"2023-10-07T23:15:26.360799+0000","last_became_peered":"2023-10-07T23:15:26.360799+0000","last_unstale":"2023-10-07T23:16:28.005469+0000","last_undegraded":"2023-10-07T23:16:28.005469+0000","last_fullsized":"2023-10-07T23:16:28.005469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:59.758856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.005566+0000","last_change":"2023-10-07T23:15:27.195615+0000","last_active":"2023-10-07T23:16:29.005566+0000","last_peered":"2023-10-07T23:16:29.005566+0000","last_clean":"2023-10-07T23:16:29.005566+0000","last_became_active":"2023-10-07T23:15:27.195312+0000","last_became_peered":"2023-10-07T23:15:27.195312+0000","last_unstale":"2023-10-07T23:16:29.005566+0000","last_undegraded":"2023-10-07T23:16:29.005566+0000","last_fullsized":"2023-10-07T23:16:29.005566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:26.361709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.001347+0000","last_change":"2023-10-07T23:15:31.392353+0000","last_active":"2023-10-07T23:16:29.001347+0000","last_peered":"2023-10-07T23:16:29.001347+0000","last_clean":"2023-10-07T23:16:29.001347+0000","last_became_active":"2023-10-07T23:15:31.391744+0000","last_became_peered":"2023-10-07T23:15:31.391744+0000","last_unstale":"2023-10-07T23:16:29.001347+0000","last_undegraded":"2023-10-07T23:16:29.001347+0000","last_fullsized":"2023-10-07T23:16:29.001347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:00.335857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008316+0000","last_change":"2023-10-07T23:15:26.362003+0000","last_active":"2023-10-07T23:16:28.008316+0000","last_peered":"2023-10-07T23:16:28.008316+0000","last_clean":"2023-10-07T23:16:28.008316+0000","last_became_active":"2023-10-07T23:15:26.358739+0000","last_became_peered":"2023-10-07T23:15:26.358739+0000","last_unstale":"2023-10-07T23:16:28.008316+0000","last_undegraded":"2023-10-07T23:16:28.008316+0000","last_fullsized":"2023-10-07T23:16:28.008316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:15:54.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002767+0000","last_change":"2023-10-07T23:15:27.204579+0000","last_active":"2023-10-07T23:16:28.002767+0000","last_peered":"2023-10-07T23:16:28.002767+0000","last_clean":"2023-10-07T23:16:28.002767+0000","last_became_active":"2023-10-07T23:15:27.204017+0000","last_became_peered":"2023-10-07T23:15:27.204017+0000","last_unstale":"2023-10-07T23:16:28.002767+0000","last_undegraded":"2023-10-07T23:16:28.002767+0000","last_fullsized":"2023-10-07T23:16:28.002767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:28:11.953641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999422+0000","last_change":"2023-10-07T23:15:31.385206+0000","last_active":"2023-10-07T23:16:27.999422+0000","last_peered":"2023-10-07T23:16:27.999422+0000","last_clean":"2023-10-07T23:16:27.999422+0000","last_became_active":"2023-10-07T23:15:31.384769+0000","last_became_peered":"2023-10-07T23:15:31.384769+0000","last_unstale":"2023-10-07T23:16:27.999422+0000","last_undegraded":"2023-10-07T23:16:27.999422+0000","last_fullsized":"2023-10-07T23:16:27.999422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:42.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006299+0000","last_change":"2023-10-07T23:15:26.363493+0000","last_active":"2023-10-07T23:16:28.006299+0000","last_peered":"2023-10-07T23:16:28.006299+0000","last_clean":"2023-10-07T23:16:28.006299+0000","last_became_active":"2023-10-07T23:15:26.363212+0000","last_became_peered":"2023-10-07T23:15:26.363212+0000","last_unstale":"2023-10-07T23:16:28.006299+0000","last_undegraded":"2023-10-07T23:16:28.006299+0000","last_fullsized":"2023-10-07T23:16:28.006299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:30:06.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.014270+0000","last_change":"2023-10-07T23:15:27.214821+0000","last_active":"2023-10-07T23:16:28.014270+0000","last_peered":"2023-10-07T23:16:28.014270+0000","last_clean":"2023-10-07T23:16:28.014270+0000","last_became_active":"2023-10-07T23:15:27.214455+0000","last_became_peered":"2023-10-07T23:15:27.214455+0000","last_unstale":"2023-10-07T23:16:28.014270+0000","last_undegraded":"2023-10-07T23:16:28.014270+0000","last_fullsized":"2023-10-07T23:16:28.014270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:47:03.630566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:27.999515+0000","last_change":"2023-10-07T23:15:31.622100+0000","last_active":"2023-10-07T23:16:27.999515+0000","last_peered":"2023-10-07T23:16:27.999515+0000","last_clean":"2023-10-07T23:16:27.999515+0000","last_became_active":"2023-10-07T23:15:31.621599+0000","last_became_peered":"2023-10-07T23:15:31.621599+0000","last_unstale":"2023-10-07T23:16:27.999515+0000","last_undegraded":"2023-10-07T23:16:27.999515+0000","last_fullsized":"2023-10-07T23:16:27.999515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:02:27.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.049631+0000","last_change":"2023-10-07T23:16:29.049214+0000","last_active":"2023-10-07T23:16:29.049631+0000","last_peered":"2023-10-07T23:16:29.049631+0000","last_clean":"2023-10-07T23:16:29.049631+0000","last_became_active":"2023-10-07T23:16:29.044778+0000","last_became_peered":"2023-10-07T23:16:29.044778+0000","last_unstale":"2023-10-07T23:16:29.049631+0000","last_undegraded":"2023-10-07T23:16:29.049631+0000","last_fullsized":"2023-10-07T23:16:29.049631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:38.061993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007511+0000","last_change":"2023-10-07T23:15:27.198819+0000","last_active":"2023-10-07T23:16:28.007511+0000","last_peered":"2023-10-07T23:16:28.007511+0000","last_clean":"2023-10-07T23:16:28.007511+0000","last_became_active":"2023-10-07T23:15:27.198543+0000","last_became_peered":"2023-10-07T23:15:27.198543+0000","last_unstale":"2023-10-07T23:16:28.007511+0000","last_undegraded":"2023-10-07T23:16:28.007511+0000","last_fullsized":"2023-10-07T23:16:28.007511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:25:39.876674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005590+0000","last_change":"2023-10-07T23:15:31.379933+0000","last_active":"2023-10-07T23:16:28.005590+0000","last_peered":"2023-10-07T23:16:28.005590+0000","last_clean":"2023-10-07T23:16:28.005590+0000","last_became_active":"2023-10-07T23:15:31.379541+0000","last_became_peered":"2023-10-07T23:15:31.379541+0000","last_unstale":"2023-10-07T23:16:28.005590+0000","last_undegraded":"2023-10-07T23:16:28.005590+0000","last_fullsized":"2023-10-07T23:16:28.005590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:49:43.102719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:17:22.983295+0000","last_change":"2023-10-07T23:15:26.362589+0000","last_active":"2023-10-07T23:17:22.983295+0000","last_peered":"2023-10-07T23:17:22.983295+0000","last_clean":"2023-10-07T23:17:22.983295+0000","last_became_active":"2023-10-07T23:15:26.362217+0000","last_became_peered":"2023-10-07T23:15:26.362217+0000","last_unstale":"2023-10-07T23:17:22.983295+0000","last_undegraded":"2023-10-07T23:17:22.983295+0000","last_fullsized":"2023-10-07T23:17:22.983295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:12.054271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007939+0000","last_change":"2023-10-07T23:15:27.200609+0000","last_active":"2023-10-07T23:16:28.007939+0000","last_peered":"2023-10-07T23:16:28.007939+0000","last_clean":"2023-10-07T23:16:28.007939+0000","last_became_active":"2023-10-07T23:15:27.200326+0000","last_became_peered":"2023-10-07T23:15:27.200326+0000","last_unstale":"2023-10-07T23:16:28.007939+0000","last_undegraded":"2023-10-07T23:16:28.007939+0000","last_fullsized":"2023-10-07T23:16:28.007939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:24.169099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012043+0000","last_change":"2023-10-07T23:15:31.395613+0000","last_active":"2023-10-07T23:16:28.012043+0000","last_peered":"2023-10-07T23:16:28.012043+0000","last_clean":"2023-10-07T23:16:28.012043+0000","last_became_active":"2023-10-07T23:15:31.395183+0000","last_became_peered":"2023-10-07T23:15:31.395183+0000","last_unstale":"2023-10-07T23:16:28.012043+0000","last_undegraded":"2023-10-07T23:16:28.012043+0000","last_fullsized":"2023-10-07T23:16:28.012043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:25.363149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005371+0000","last_change":"2023-10-07T23:15:26.363944+0000","last_active":"2023-10-07T23:16:28.005371+0000","last_peered":"2023-10-07T23:16:28.005371+0000","last_clean":"2023-10-07T23:16:28.005371+0000","last_became_active":"2023-10-07T23:15:26.363634+0000","last_became_peered":"2023-10-07T23:15:26.363634+0000","last_unstale":"2023-10-07T23:16:28.005371+0000","last_undegraded":"2023-10-07T23:16:28.005371+0000","last_fullsized":"2023-10-07T23:16:28.005371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:41.540528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.007323+0000","last_change":"2023-10-07T23:15:27.204168+0000","last_active":"2023-10-07T23:16:28.007323+0000","last_peered":"2023-10-07T23:16:28.007323+0000","last_clean":"2023-10-07T23:16:28.007323+0000","last_became_active":"2023-10-07T23:15:27.203956+0000","last_became_peered":"2023-10-07T23:15:27.203956+0000","last_unstale":"2023-10-07T23:16:28.007323+0000","last_undegraded":"2023-10-07T23:16:28.007323+0000","last_fullsized":"2023-10-07T23:16:28.007323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:10.968685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003056+0000","last_change":"2023-10-07T23:15:31.379973+0000","last_active":"2023-10-07T23:16:28.003056+0000","last_peered":"2023-10-07T23:16:28.003056+0000","last_clean":"2023-10-07T23:16:28.003056+0000","last_became_active":"2023-10-07T23:15:31.379578+0000","last_became_peered":"2023-10-07T23:15:31.379578+0000","last_unstale":"2023-10-07T23:16:28.003056+0000","last_undegraded":"2023-10-07T23:16:28.003056+0000","last_fullsized":"2023-10-07T23:16:28.003056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:30.351098+0000","last_clean_scrub_stamp":"2023-10-07T23:15:30.351098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:48.037875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.051993+0000","last_change":"2023-10-07T23:16:29.051993+0000","last_active":"2023-10-07T23:16:29.051993+0000","last_peered":"2023-10-07T23:16:29.051993+0000","last_clean":"2023-10-07T23:16:29.051993+0000","last_became_active":"2023-10-07T23:16:29.046123+0000","last_became_peered":"2023-10-07T23:16:29.046123+0000","last_unstale":"2023-10-07T23:16:29.051993+0000","last_undegraded":"2023-10-07T23:16:29.051993+0000","last_fullsized":"2023-10-07T23:16:29.051993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:25.323358+0000","last_clean_scrub_stamp":"2023-10-07T23:15:25.323358+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:52.174959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008468+0000","last_change":"2023-10-07T23:15:27.200009+0000","last_active":"2023-10-07T23:16:28.008468+0000","last_peered":"2023-10-07T23:16:28.008468+0000","last_clean":"2023-10-07T23:16:28.008468+0000","last_became_active":"2023-10-07T23:15:27.199723+0000","last_became_peered":"2023-10-07T23:15:27.199723+0000","last_unstale":"2023-10-07T23:16:28.008468+0000","last_undegraded":"2023-10-07T23:16:28.008468+0000","last_fullsized":"2023-10-07T23:16:28.008468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:55.020725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.002713+0000","last_change":"2023-10-07T23:15:27.204333+0000","last_active":"2023-10-07T23:16:28.002713+0000","last_peered":"2023-10-07T23:16:28.002713+0000","last_clean":"2023-10-07T23:16:28.002713+0000","last_became_active":"2023-10-07T23:15:27.203607+0000","last_became_peered":"2023-10-07T23:15:27.203607+0000","last_unstale":"2023-10-07T23:16:28.002713+0000","last_undegraded":"2023-10-07T23:16:28.002713+0000","last_fullsized":"2023-10-07T23:16:28.002713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:43.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006794+0000","last_change":"2023-10-07T23:15:27.202832+0000","last_active":"2023-10-07T23:16:28.006794+0000","last_peered":"2023-10-07T23:16:28.006794+0000","last_clean":"2023-10-07T23:16:28.006794+0000","last_became_active":"2023-10-07T23:15:27.202611+0000","last_became_peered":"2023-10-07T23:15:27.202611+0000","last_unstale":"2023-10-07T23:16:28.006794+0000","last_undegraded":"2023-10-07T23:16:28.006794+0000","last_fullsized":"2023-10-07T23:16:28.006794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:00.091376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.006214+0000","last_change":"2023-10-07T23:15:27.205851+0000","last_active":"2023-10-07T23:16:28.006214+0000","last_peered":"2023-10-07T23:16:28.006214+0000","last_clean":"2023-10-07T23:16:28.006214+0000","last_became_active":"2023-10-07T23:15:27.205571+0000","last_became_peered":"2023-10-07T23:15:27.205571+0000","last_unstale":"2023-10-07T23:16:28.006214+0000","last_undegraded":"2023-10-07T23:16:28.006214+0000","last_fullsized":"2023-10-07T23:16:28.006214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:06:18.928602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.012237+0000","last_change":"2023-10-07T23:15:27.206961+0000","last_active":"2023-10-07T23:16:28.012237+0000","last_peered":"2023-10-07T23:16:28.012237+0000","last_clean":"2023-10-07T23:16:28.012237+0000","last_became_active":"2023-10-07T23:15:27.206670+0000","last_became_peered":"2023-10-07T23:15:27.206670+0000","last_unstale":"2023-10-07T23:16:28.012237+0000","last_undegraded":"2023-10-07T23:16:28.012237+0000","last_fullsized":"2023-10-07T23:16:28.012237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:07.505584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-07T23:16:29.044932+0000","last_change":"2023-10-07T23:16:29.044932+0000","last_active":"2023-10-07T23:16:29.044932+0000","last_peered":"2023-10-07T23:16:29.044932+0000","last_clean":"2023-10-07T23:16:29.044932+0000","last_became_active":"2023-10-07T23:16:29.044454+0000","last_became_peered":"2023-10-07T23:16:29.044454+0000","last_unstale":"2023-10-07T23:16:29.044932+0000","last_undegraded":"2023-10-07T23:16:29.044932+0000","last_fullsized":"2023-10-07T23:16:29.044932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:37:55.975049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003230+0000","last_change":"2023-10-07T23:15:27.199808+0000","last_active":"2023-10-07T23:16:28.003230+0000","last_peered":"2023-10-07T23:16:28.003230+0000","last_clean":"2023-10-07T23:16:28.003230+0000","last_became_active":"2023-10-07T23:15:27.198770+0000","last_became_peered":"2023-10-07T23:15:27.198770+0000","last_unstale":"2023-10-07T23:16:28.003230+0000","last_undegraded":"2023-10-07T23:16:28.003230+0000","last_fullsized":"2023-10-07T23:16:28.003230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:44.288056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008782+0000","last_change":"2023-10-07T23:15:27.196765+0000","last_active":"2023-10-07T23:16:28.008782+0000","last_peered":"2023-10-07T23:16:28.008782+0000","last_clean":"2023-10-07T23:16:28.008782+0000","last_became_active":"2023-10-07T23:15:27.196466+0000","last_became_peered":"2023-10-07T23:15:27.196466+0000","last_unstale":"2023-10-07T23:16:28.008782+0000","last_undegraded":"2023-10-07T23:16:28.008782+0000","last_fullsized":"2023-10-07T23:16:28.008782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:29:11.820303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.015307+0000","last_change":"2023-10-07T23:15:27.216388+0000","last_active":"2023-10-07T23:16:28.015307+0000","last_peered":"2023-10-07T23:16:28.015307+0000","last_clean":"2023-10-07T23:16:28.015307+0000","last_became_active":"2023-10-07T23:15:27.216013+0000","last_became_peered":"2023-10-07T23:15:27.216013+0000","last_unstale":"2023-10-07T23:16:28.015307+0000","last_undegraded":"2023-10-07T23:16:28.015307+0000","last_fullsized":"2023-10-07T23:16:28.015307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:36:52.048738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003934+0000","last_change":"2023-10-07T23:15:27.204855+0000","last_active":"2023-10-07T23:16:28.003934+0000","last_peered":"2023-10-07T23:16:28.003934+0000","last_clean":"2023-10-07T23:16:28.003934+0000","last_became_active":"2023-10-07T23:15:27.204579+0000","last_became_peered":"2023-10-07T23:15:27.204579+0000","last_unstale":"2023-10-07T23:16:28.003934+0000","last_undegraded":"2023-10-07T23:16:28.003934+0000","last_fullsized":"2023-10-07T23:16:28.003934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:44:32.793999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.008400+0000","last_change":"2023-10-07T23:15:27.198945+0000","last_active":"2023-10-07T23:16:28.008400+0000","last_peered":"2023-10-07T23:16:28.008400+0000","last_clean":"2023-10-07T23:16:28.008400+0000","last_became_active":"2023-10-07T23:15:27.198601+0000","last_became_peered":"2023-10-07T23:15:27.198601+0000","last_unstale":"2023-10-07T23:16:28.008400+0000","last_undegraded":"2023-10-07T23:16:28.008400+0000","last_fullsized":"2023-10-07T23:16:28.008400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:01:20.775299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.005732+0000","last_change":"2023-10-07T23:15:27.206050+0000","last_active":"2023-10-07T23:16:28.005732+0000","last_peered":"2023-10-07T23:16:28.005732+0000","last_clean":"2023-10-07T23:16:28.005732+0000","last_became_active":"2023-10-07T23:15:27.205774+0000","last_became_peered":"2023-10-07T23:15:27.205774+0000","last_unstale":"2023-10-07T23:16:28.005732+0000","last_undegraded":"2023-10-07T23:16:28.005732+0000","last_fullsized":"2023-10-07T23:16:28.005732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:49.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.000267+0000","last_change":"2023-10-07T23:15:27.205193+0000","last_active":"2023-10-07T23:16:28.000267+0000","last_peered":"2023-10-07T23:16:28.000267+0000","last_clean":"2023-10-07T23:16:28.000267+0000","last_became_active":"2023-10-07T23:15:27.204845+0000","last_became_peered":"2023-10-07T23:15:27.204845+0000","last_unstale":"2023-10-07T23:16:28.000267+0000","last_undegraded":"2023-10-07T23:16:28.000267+0000","last_fullsized":"2023-10-07T23:16:28.000267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:30.167000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-07T23:16:28.003067+0000","last_change":"2023-10-07T23:15:27.200238+0000","last_active":"2023-10-07T23:16:28.003067+0000","last_peered":"2023-10-07T23:16:28.003067+0000","last_clean":"2023-10-07T23:16:28.003067+0000","last_became_active":"2023-10-07T23:15:27.199946+0000","last_became_peered":"2023-10-07T23:15:27.199946+0000","last_unstale":"2023-10-07T23:16:28.003067+0000","last_undegraded":"2023-10-07T23:16:28.003067+0000","last_fullsized":"2023-10-07T23:16:28.003067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:15:26.330052+0000","last_clean_scrub_stamp":"2023-10-07T23:15:26.330052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:24:45.931624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":964,"num_read_kb":503,"num_write":211,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383715,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":495361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.624,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.622,"15min":0.621},"min":{"1min":0.42999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.624,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.755,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.46200000000000002,"15min":0.45600000000000002},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72599999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.53300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90000000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.83299999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.002,"15min":1.002},"last":0.42399999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89200000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0189999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.57399999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.323,"15min":1.323},"last":0.81699999999999995}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.751}]},{"osd":9,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.68000000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1140000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78100000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76800000000000002,"15min":0.76000000000000001},"min":{"1min":0.59599999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0940000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.61899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82899999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298908,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446244,"statfs":{"total":95995035648,"available":95688953856,"internally_reserved":0,"allocated":1658880,"data_stored":1089923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48299999999999998,"15min":0.48799999999999999},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.874,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94399999999999995,"5min":1.159,"15min":1.159},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.51300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.5,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.141,"5min":1.214,"15min":1.214},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Oct 7 23:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.55300000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.026,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0149999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76400000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.84299999999999997,"5min":1.171,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.86899999999999999,"15min":0.92700000000000005},"min":{"1min":0.625,"5min":0.502,"15min":0.502},"max":{"1min":0.871,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97299999999999998,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.752,"15min":0.75},"min":{"1min":0.628,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95899999999999996,"5min":1.179,"15min":1.179},"last":0.78100000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808879,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72799999999999998,"15min":0.76000000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.68200000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.054,"15min":1.054},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68700000000000006,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.495}]},{"osd":2,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.82099999999999995,"15min":0.85299999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.77300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.278,"15min":1.278},"last":0.81699999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.622},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80800000000000005,"15min":0.84299999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.63700000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79300000000000004,"15min":0.84699999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.345,"15min":1.345},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.82199999999999995,"15min":0.86399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.421,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.85899999999999999,"15min":0.91300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0860000000000001,"5min":1.579,"15min":1.579},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.98099999999999998},"min":{"1min":0.54000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.548,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:14:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.95399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97699999999999998,"15min":1.0289999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.5880000000000001,"5min":1.603,"15min":1.603},"last":0.76200000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.97399999999999998,"15min":1.0389999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2470000000000001,"5min":1.782,"15min":1.782},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.92600000000000005,"15min":0.98499999999999999},"min":{"1min":0.60299999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6850000000000001,"5min":1.956,"15min":1.956},"last":0.82899999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.97299999999999998,"15min":1.0129999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.647},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.94799999999999995,"15min":1.002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94399999999999995,"5min":1.663,"15min":1.663},"last":0.89100000000000001}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.77200000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":1044480,"data_stored":495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.85299999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.753,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.438}]},{"osd":3,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64500000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67800000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52000000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.59299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.022,"5min":1.597,"15min":1.597},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.69499999999999995,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1,"15min":1},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.80500000000000005,"15min":0.79400000000000004},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4259999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.64600000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.755}]},{"osd":10,"last update":"Sat Oct 7 23:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76100000000000001,"15min":0.73299999999999998},"min":{"1min":0.61899999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":1.008,"15min":1.008},"last":0.82099999999999995}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.84099999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298888,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446264,"statfs":{"total":95995035648,"available":95688974336,"internally_reserved":0,"allocated":1638400,"data_stored":1087813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57499999999999996,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42499999999999999,"15min":0.41099999999999998},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61499999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.311,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.504}]},{"osd":2,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59299999999999997,"15min":0.56699999999999995},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":1.51,"15min":1.51},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.496,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65900000000000003,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92900000000000005,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85199999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.47099999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.92800000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Sat Oct 7 23:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0329999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.74199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94599999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58199999999999996}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99399999999999999,"5min":1.056,"15min":1.056},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.52500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.002,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.754},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76400000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":1024000,"data_stored":494596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.752},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.65800000000000003},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.653,"5min":6.5700000000000003,"15min":6.5700000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.45500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.439},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.46500000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.95999999999999996}]},{"osd":5,"last update":"Sat Oct 7 23:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1299999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.873}]},{"osd":7,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.82099999999999995,"15min":0.78700000000000003},"min":{"1min":0.58099999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":0.879},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.677,"5min":6.524,"15min":6.524},"last":0.86199999999999999}]},{"osd":8,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.089,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88600000000000001,"15min":0.88900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":2.8780000000000001},"last":0.82299999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.61099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95999999999999996,"15min":0.97799999999999998},"min":{"1min":0.65800000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.91900000000000004}]},{"osd":10,"last update":"Sat Oct 7 23:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.111,"15min":1.1890000000000001},"min":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.911,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0640000000000001,"15min":1.1299999999999999},"min":{"1min":0.68999999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.294,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":0.90700000000000003}]},{"osd":11,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":499285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.51800000000000002},"min":{"1min":0.47699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.60999999999999999,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.012,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90900000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.59499999999999997},"min":{"1min":0.51400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.84299999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.47199999999999998,"15min":0.41599999999999998},"min":{"1min":0.29899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82599999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.433},"min":{"1min":0.33800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Sat Oct 7 23:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65100000000000002,"15min":0.56200000000000006},"min":{"1min":0.498,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.52100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.70799999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":1.004,"15min":1.004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89800000000000002,"5min":1.107,"15min":1.107},"last":0.624},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63900000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80500000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0069999999999999,"5min":1.131,"15min":1.131},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76700000000000002,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94699999999999995,"5min":1.075,"15min":1.075},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Oct 7 23:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.752},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.89000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298312,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446840,"statfs":{"total":95995035648,"available":95689564160,"internally_reserved":0,"allocated":1048576,"data_stored":495996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.47399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.70399999999999996,"15min":0.57399999999999995},"min":{"1min":0.57099999999999995,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":5.4249999999999998,"5min":5.4249999999999998,"15min":5.4249999999999998},"last":0.65100000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.73499999999999999,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.5990000000000002,"5min":6.5990000000000002,"15min":6.5990000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.879,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.625,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.76300000000000001,"15min":0.66300000000000003},"min":{"1min":0.55400000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":5.4669999999999996,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68799999999999994,"15min":0.65400000000000003},"min":{"1min":0.51300000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0529999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.81399999999999995,"15min":0.77900000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:16:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.88300000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79800000000000004,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.67900000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Oct 7 23:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.49399999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.6180000000000003,"5min":5.6180000000000003,"15min":5.6180000000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89200000000000002,"15min":0.875},"min":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":5.0609999999999999,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1339999999999999,"5min":1.381,"15min":1.381},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0049999999999999,"15min":0.97999999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":5.5439999999999996,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.86699999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.88100000000000001,"15min":0.86899999999999999},"min":{"1min":0.68400000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.0029999999999999,"15min":0.94399999999999995},"min":{"1min":0.79300000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":6.6539999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.93600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298316,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446836,"statfs":{"total":95995035648,"available":95689560064,"internally_reserved":0,"allocated":1052672,"data_stored":499485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.54100000000000004},"min":{"1min":0.48899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.51000000000000001},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.29899999999999999},"max":{"1min":0.88200000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.373,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Oct 7 23:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84999999999999998,"5min":0.871,"15min":0.871},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.746},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.45200000000000001},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.29699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42799999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88800000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.57399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85699999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63500000000000001}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.628,"15min":0.60099999999999998},"min":{"1min":0.59099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56999999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.58599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.65100000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.96899999999999997,"5min":1.024,"15min":1.024},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Oct 7 23:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.75700000000000001,"15min":0.71399999999999997},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.03,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.77300000000000002,"15min":0.73099999999999998},"min":{"1min":0.627,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.044,"5min":1.147,"15min":1.147},"last":0.76300000000000001}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":1040384,"data_stored":494675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"min":{"1min":0.34499999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80500000000000005,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.0169999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70899999999999996,"15min":0.629},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92400000000000004,"5min":2.387,"15min":2.387},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0680000000000001,"5min":1.218,"15min":1.218},"last":0.57399999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71699999999999997,"15min":0.64000000000000001},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95699999999999996,"5min":1.278,"15min":1.278},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.68600000000000005},"min":{"1min":0.58599999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.01,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.139,"5min":0.80300000000000005,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":6.3940000000000001},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67200000000000004},"min":{"1min":0.46899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Oct 7 23:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.64600000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.89200000000000002,"15min":0.81799999999999995},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":6.4160000000000004,"5min":6.4160000000000004,"15min":6.4160000000000004},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Oct 7 23:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Oct 7 23:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.93400000000000005,"15min":0.93200000000000005},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.002,"5min":0.998,"15min":1.0169999999999999},"min":{"1min":0.73199999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Sat Oct 7 23:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.96099999999999997,"15min":0.99399999999999999},"min":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0269999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.95299999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.82999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":1032192,"data_stored":495155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.40300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.495},"min":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56699999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41299999999999998,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.59199999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56599999999999995,"15min":0.54400000000000004},"min":{"1min":0.45300000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.55000000000000004},"min":{"1min":0.51000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.88,"15min":0.88},"last":0.66500000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.53800000000000003},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.498}]},{"osd":5,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.56499999999999995},"min":{"1min":0.52100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.64900000000000002,"15min":0.56899999999999995},"min":{"1min":0.60099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66800000000000004,"15min":0.64100000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91500000000000004,"5min":1.004,"15min":1.004},"last":0.57499999999999996}]},{"osd":7,"last update":"Sat Oct 7 23:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83699999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002}]},{"osd":8,"last update":"Sat Oct 7 23:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80400000000000005}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.81200000000000006,"15min":0.79000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.95899999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.755},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.71199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69899999999999995}]},{"osd":10,"last update":"Sat Oct 7 23:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94099999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Oct 7 23:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.89400000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298896,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446256,"statfs":{"total":95995035648,"available":95688966144,"internally_reserved":0,"allocated":1646592,"data_stored":1086978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.6230000000000002,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79800000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.503},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.50600000000000001},"min":{"1min":0.436,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71499999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41699999999999998,"15min":0.36899999999999999},"min":{"1min":0.36399999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.63200000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.373},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.38600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":3.827,"15min":3.827},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.622,"15min":0.55200000000000005},"min":{"1min":0.46400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90800000000000003,"5min":3.903,"15min":3.903},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.54600000000000004},"min":{"1min":0.52200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60799999999999998,"15min":0.55800000000000005},"min":{"1min":0.50800000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Oct 7 23:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.89600000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Oct 7 23:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54500000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.55200000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.749}]},{"osd":8,"last update":"Sat Oct 7 23:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.47899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.89300000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62}]},{"osd":9,"last update":"Sat Oct 7 23:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.63400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.747,"15min":0.748},"min":{"1min":0.61899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.90500000000000003,"5min":1,"15min":1},"last":0.59999999999999998}]},{"osd":10,"last update":"Sat Oct 7 23:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.70899999999999996},"min":{"1min":0.65700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.625,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:17:29.862 INFO:tasks.fwd_scrub.ceph_manager:PG 3.1d is not active+clean 2023-10-07T23:17:29.863 INFO:tasks.fwd_scrub.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'peering', 'last_fresh': '2023-10-07T23:17:28.058734+0000', 'last_change': '2023-10-07T23:17:28.058734+0000', 'last_active': '0.000000', 'last_peered': '0.000000', 'last_clean': '0.000000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:17:28.058734+0000', 'last_undegraded': '2023-10-07T23:17:28.058734+0000', 'last_fullsized': '2023-10-07T23:17:28.058734+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 84, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:15:26.330052+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:15:26.330052+0000', 'last_clean_scrub_stamp': '2023-10-07T23:15:26.330052+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-08T23:45:18.855595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 4, 1], 'acting': [9, 4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-10-07T23:17:29.863 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/fwd_scrub.py", line 134, in task assert manager.is_clean() AssertionError 2023-10-07T23:17:29.957 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1e7a0e5812964784baf5224c7ca88154 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/fwd_scrub.py", line 134, in task assert manager.is_clean() AssertionError 2023-10-07T23:17:29.958 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-07T23:17:29.967 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-07T23:17:29.983 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-07T23:17:29.983 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-07T23:17:29.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:17:29.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-07T23:17:30.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T23:17:30.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:30.093 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T23:17:30.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:30.130 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-07T23:17:30.207 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T23:17:30.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:30.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:17:30.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:30 smithi067 ceph-mon[82723]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:30.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:30 smithi067 ceph-mon[82723]: osdmap e93: 12 total, 12 up, 12 in 2023-10-07T23:17:30.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:30 smithi067 ceph-mon[82723]: from='client.25405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:30.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:17:30.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-10-07T23:17:30.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:30 smithi123 ceph-mon[122926]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:30.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:30 smithi123 ceph-mon[122926]: osdmap e93: 12 total, 12 up, 12 in 2023-10-07T23:17:30.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:30 smithi123 ceph-mon[122926]: from='client.25405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:30.444 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-07T23:17:30.454 INFO:tasks.cephadm:Teardown begin 2023-10-07T23:17:30.454 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T23:17:30.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:17:30.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:30 smithi086 ceph-mon[122131]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:30.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:30 smithi086 ceph-mon[122131]: osdmap e93: 12 total, 12 up, 12 in 2023-10-07T23:17:30.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:30 smithi086 ceph-mon[122131]: from='client.25405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3433879628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fdef0 msgr2=0x7f75ac0fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3433879628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fdef0 0x7f75ac0fe340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f75980049c0 tx=0x7f7598009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3433879628 shutdown_connections 2023-10-07T23:17:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3433879628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fcce0 0x7f75ac0fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3433879628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75ac0fe880 0x7f75ac105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3433879628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fdef0 0x7f75ac0fe340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3433879628 >> 172.21.15.67:0/3433879628 conn(0x7f75ac0f8460 msgr2=0x7f75ac0fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3433879628 shutdown_connections 2023-10-07T23:17:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.646+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3433879628 wait complete. 2023-10-07T23:17:30.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 Processor -- start 2023-10-07T23:17:30.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 -- start start 2023-10-07T23:17:30.650 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75ac0fcce0 0x7f75ac10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 0x7f75ac10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fe880 0x7f75ac114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac115540 con 0x7f75ac0fe880 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac1156b0 con 0x7f75ac0fdef0 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75b1d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac115870 con 0x7f75ac0fcce0 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.647+0000 7f75abfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 0x7f75ac10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 0x7f75ac10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:40524/0 (socket says 172.21.15.67:40524) 2023-10-07T23:17:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 -- 172.21.15.67:0/3016492091 learned_addr learned my addr 172.21.15.67:0/3016492091 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75b0d9b700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75ac0fcce0 0x7f75ac10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75b159c700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fe880 0x7f75ac114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 -- 172.21.15.67:0/3016492091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75ac0fcce0 msgr2=0x7f75ac10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75ac0fcce0 0x7f75ac10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 -- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fe880 msgr2=0x7f75ac114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fe880 0x7f75ac114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 -- 172.21.15.67:0/3016492091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598005040 con 0x7f75ac0fdef0 2023-10-07T23:17:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75abfff700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 0x7f75ac10dea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f759800b0a0 tx=0x7f7598009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:30.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759800e050 con 0x7f75ac0fdef0 2023-10-07T23:17:30.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac115af0 con 0x7f75ac0fdef0 2023-10-07T23:17:30.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.648+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f759801b3f0 con 0x7f75ac0fdef0 2023-10-07T23:17:30.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.649+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598012630 con 0x7f75ac0fdef0 2023-10-07T23:17:30.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.649+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac115fd0 con 0x7f75ac0fdef0 2023-10-07T23:17:30.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.650+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ac103ba0 con 0x7f75ac0fdef0 2023-10-07T23:17:30.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.652+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f759801b7f0 con 0x7f75ac0fdef0 2023-10-07T23:17:30.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.653+0000 7f75a97fa700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f759c09b770 0x7f759c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:30.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.653+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9103+0+0 (secure 0 0 0) 0x7f75980c6e10 con 0x7f75ac0fdef0 2023-10-07T23:17:30.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.653+0000 7f75b0d9b700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f759c09b770 0x7f759c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.654+0000 7f75b0d9b700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f759c09b770 0x7f759c09dc20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009b80 tx=0x7f75a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.655+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f7598092270 con 0x7f75ac0fdef0 2023-10-07T23:17:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.909+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75ac108520 con 0x7f75ac0fdef0 2023-10-07T23:17:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.910+0000 7f75a97fa700 1 -- 172.21.15.67:0/3016492091 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f7598095890 con 0x7f75ac0fdef0 2023-10-07T23:17:30.910 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:30.911 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:17:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f759c09b770 msgr2=0x7f759c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f759c09b770 0x7f759c09dc20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009b80 tx=0x7f75a000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 msgr2=0x7f75ac10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 0x7f75ac10dea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f759800b0a0 tx=0x7f7598009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 shutdown_connections 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f759c09b770 0x7f759c09dc20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75ac0fe880 0x7f75ac114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75ac0fcce0 0x7f75ac10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 --2- 172.21.15.67:0/3016492091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75ac0fdef0 0x7f75ac10dea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 >> 172.21.15.67:0/3016492091 conn(0x7f75ac0f8460 msgr2=0x7f75ac0fa660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 shutdown_connections 2023-10-07T23:17:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:30.912+0000 7f75b1d9d700 1 -- 172.21.15.67:0/3016492091 wait complete. 2023-10-07T23:17:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:17:30.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:17:31.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.120+0000 7f4db3fff700 1 -- 172.21.15.67:0/346321850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4071bc0 msgr2=0x7f4db4070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.120+0000 7f4db3fff700 1 --2- 172.21.15.67:0/346321850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4071bc0 0x7f4db4070030 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4da40049c0 tx=0x7f4da4009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 -- 172.21.15.67:0/346321850 shutdown_connections 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 --2- 172.21.15.67:0/346321850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db4104b80 0x7f4db4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 --2- 172.21.15.67:0/346321850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4071bc0 0x7f4db4070030 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 --2- 172.21.15.67:0/346321850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db4070570 0x7f4db40709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 -- 172.21.15.67:0/346321850 >> 172.21.15.67:0/346321850 conn(0x7f4db40f8460 msgr2=0x7f4db40fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:31.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 -- 172.21.15.67:0/346321850 shutdown_connections 2023-10-07T23:17:31.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 -- 172.21.15.67:0/346321850 wait complete. 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.121+0000 7f4db3fff700 1 Processor -- start 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 -- start start 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db4070570 0x7f4db410fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db4071bc0 0x7f4db4110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 0x7f4db41171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41177e0 con 0x7f4db4104b80 2023-10-07T23:17:31.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4117950 con 0x7f4db4070570 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4117b10 con 0x7f4db4071bc0 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 0x7f4db41171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 0x7f4db41171a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46248/0 (socket says 172.21.15.67:46248) 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 -- 172.21.15.67:0/1604609275 learned_addr learned my addr 172.21.15.67:0/1604609275 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db2ffd700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db4070570 0x7f4db410fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db27fc700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db4071bc0 0x7f4db4110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 -- 172.21.15.67:0/1604609275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db4071bc0 msgr2=0x7f4db4110150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db4071bc0 0x7f4db4110150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 -- 172.21.15.67:0/1604609275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db4070570 msgr2=0x7f4db410fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.122+0000 7f4db37fe700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db4070570 0x7f4db410fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4db37fe700 1 -- 172.21.15.67:0/1604609275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4005040 con 0x7f4db4104b80 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4db37fe700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 0x7f4db41171a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00ec30 tx=0x7f4dac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac007b30 con 0x7f4db4104b80 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db4117df0 con 0x7f4db4104b80 2023-10-07T23:17:31.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dac00f040 con 0x7f4db4104b80 2023-10-07T23:17:31.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00ca30 con 0x7f4db4104b80 2023-10-07T23:17:31.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.123+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db4118280 con 0x7f4db4104b80 2023-10-07T23:17:31.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.124+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db41032d0 con 0x7f4db4104b80 2023-10-07T23:17:31.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.126+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f4dac01d400 con 0x7f4db4104b80 2023-10-07T23:17:31.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.127+0000 7f4d9b7fe700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d9c09b770 0x7f4d9c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.127+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9103+0+0 (secure 0 0 0) 0x7f4dac0c5740 con 0x7f4db4104b80 2023-10-07T23:17:31.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.127+0000 7f4db2ffd700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d9c09b770 0x7f4d9c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.128+0000 7f4db2ffd700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d9c09b770 0x7f4d9c09dc20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4da4004890 tx=0x7f4da4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:31.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.131+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f4dac090ba0 con 0x7f4db4104b80 2023-10-07T23:17:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:31 smithi067 ceph-mon[82723]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:31 smithi067 ceph-mon[82723]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-07T23:17:31.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:31 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/3016492091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:31.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.380+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4db410a780 con 0x7f4db4104b80 2023-10-07T23:17:31.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.381+0000 7f4d9b7fe700 1 -- 172.21.15.67:0/1604609275 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10779 (secure 0 0 0) 0x7f4dac00ce60 con 0x7f4db4104b80 2023-10-07T23:17:31.382 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:31.382 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14421,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/959405817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":959405817},{"type":"v1","addr":"172.21.15.67:6835","nonce":959405817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/3904846086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":3904846086},{"type":"v1","addr":"172.21.15.67:6839","nonce":3904846086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-07T23:15:26.869397+0000","modified":"2023-10-07T23:15:55.854427+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24538,"mds_2":24521,"mds_3":24467,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/3010063667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":3010063667},{"type":"v1","addr":"172.21.15.67:6841","nonce":3010063667}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1133310774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1133310774},{"type":"v1","addr":"172.21.15.86:6833","nonce":1133310774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":18,"addr":"172.21.15.123:6833/1348503065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1348503065},{"type":"v1","addr":"172.21.15.123:6833","nonce":1348503065}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"d","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.67:6837/909177824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":909177824},{"type":"v1","addr":"172.21.15.67:6837","nonce":909177824}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/3322109112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3322109112},{"type":"v1","addr":"172.21.15.86:6835","nonce":3322109112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/4209275840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4209275840},{"type":"v1","addr":"172.21.15.123:6835","nonce":4209275840}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1647886755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1647886755},{"type":"v1","addr":"172.21.15.123:6837","nonce":1647886755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.86:6837/4272295866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":4272295866},{"type":"v1","addr":"172.21.15.86:6837","nonce":4272295866}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6839/3252673889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":3252673889},{"type":"v1","addr":"172.21.15.123:6839","nonce":3252673889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6839/2147538694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2147538694},{"type":"v1","addr":"172.21.15.86:6839","nonce":2147538694}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:17:31.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d9c09b770 msgr2=0x7f4d9c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d9c09b770 0x7f4d9c09dc20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4da4004890 tx=0x7f4da4009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 msgr2=0x7f4db41171a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 0x7f4db41171a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00ec30 tx=0x7f4dac00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 shutdown_connections 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db4070570 0x7f4db410fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f4d9c09b770 0x7f4d9c09dc20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db4104b80 0x7f4db41171a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 --2- 172.21.15.67:0/1604609275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4db4071bc0 0x7f4db4110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 >> 172.21.15.67:0/1604609275 conn(0x7f4db40f8460 msgr2=0x7f4db40fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 shutdown_connections 2023-10-07T23:17:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.384+0000 7f4db3fff700 1 -- 172.21.15.67:0/1604609275 wait complete. 2023-10-07T23:17:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2023-10-07T23:17:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:31 smithi123 ceph-mon[122926]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:31 smithi123 ceph-mon[122926]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-07T23:17:31.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:31 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/3016492091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:31.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:17:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:31 smithi086 ceph-mon[122131]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-07T23:17:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:31 smithi086 ceph-mon[122131]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-10-07T23:17:31.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:31 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/3016492091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.592+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/926865708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc0fefc0 msgr2=0x7f3ccc0ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.592+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/926865708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc0ff3d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00049c0 tx=0x7f3cc0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/926865708 shutdown_connections 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/926865708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc100b60 0x7f3ccc107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/926865708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc0ff3d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/926865708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc1001d0 0x7f3ccc100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/926865708 >> 172.21.15.67:0/926865708 conn(0x7f3ccc0fa630 msgr2=0x7f3ccc0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/926865708 shutdown_connections 2023-10-07T23:17:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/926865708 wait complete. 2023-10-07T23:17:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.593+0000 7f3cd1d9d700 1 Processor -- start 2023-10-07T23:17:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 -- start start 2023-10-07T23:17:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc10fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1001d0 0x7f3ccc1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc100b60 0x7f3ccc117170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc117780 con 0x7f3ccc100b60 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1178f0 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd1d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc117ab0 con 0x7f3ccc1001d0 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd159c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc100b60 0x7f3ccc117170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd159c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc100b60 0x7f3ccc117170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46276/0 (socket says 172.21.15.67:46276) 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd159c700 1 -- 172.21.15.67:0/2462822246 learned_addr learned my addr 172.21.15.67:0/2462822246 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3cd0d9b700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc10fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.594+0000 7f3ccbfff700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1001d0 0x7f3ccc1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd0d9b700 1 -- 172.21.15.67:0/2462822246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1001d0 msgr2=0x7f3ccc1100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd0d9b700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1001d0 0x7f3ccc1100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd0d9b700 1 -- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc100b60 msgr2=0x7f3ccc117170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd0d9b700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc100b60 0x7f3ccc117170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd0d9b700 1 -- 172.21.15.67:0/2462822246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0005040 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3ccbfff700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1001d0 0x7f3ccc1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd0d9b700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc10fba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00049c0 tx=0x7f3cc0009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000e050 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc117d30 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc001b3f0 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0012630 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.595+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc118190 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.596+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc10a820 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.598+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f3cc0012a60 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.599+0000 7f3cc97fa700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3cbc09b770 0x7f3cbc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.600+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9103+0+0 (secure 0 0 0) 0x7f3cc00c6a30 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.600+0000 7f3ccbfff700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3cbc09b770 0x7f3cbc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.600+0000 7f3ccbfff700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3cbc09b770 0x7f3cbc09dc20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8004600 tx=0x7f3cb800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:31.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.602+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f3cc0091e90 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.807+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ccc105e30 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.809+0000 7f3cc97fa700 1 -- 172.21.15.67:0/2462822246 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+23776 (secure 0 0 0) 0x7f3cc0028090 con 0x7f3ccc0fefc0 2023-10-07T23:17:31.809 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-07T23:17:31.810 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":93,"fsid":"227de690-6566-11ee-8db5-212e2dc638e7","created":"2023-10-07T23:07:33.436637+0000","modified":"2023-10-07T23:17:29.051549+0000","last_up_change":"2023-10-07T23:14:38.141173+0000","last_in_change":"2023-10-07T23:14:18.947288+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:10:50.442097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:15:25.227955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:15:26.323230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-07T23:15:29.861861+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a7d0d95f-5162-4fdd-8f39-a33aa62127a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6803","nonce":1635424780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6805","nonce":1635424780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6809","nonce":1635424780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1635424780},{"type":"v1","addr":"172.21.15.67:6807","nonce":1635424780}]},"public_addr":"172.21.15.67:6803/1635424780","cluster_addr":"172.21.15.67:6805/1635424780","heartbeat_back_addr":"172.21.15.67:6809/1635424780","heartbeat_front_addr":"172.21.15.67:6807/1635424780","state":["exists","up"]},{"osd":1,"uuid":"04b0f276-f836-435c-88ac-d6832505fc36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6801","nonce":3463493653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6803","nonce":3463493653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6807","nonce":3463493653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3463493653},{"type":"v1","addr":"172.21.15.86:6805","nonce":3463493653}]},"public_addr":"172.21.15.86:6801/3463493653","cluster_addr":"172.21.15.86:6803/3463493653","heartbeat_back_addr":"172.21.15.86:6807/3463493653","heartbeat_front_addr":"172.21.15.86:6805/3463493653","state":["exists","up"]},{"osd":2,"uuid":"8cbf72c2-7a4a-43ab-8a14-8cba2f6f0135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6801","nonce":3773700997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6803","nonce":3773700997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6807","nonce":3773700997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3773700997},{"type":"v1","addr":"172.21.15.123:6805","nonce":3773700997}]},"public_addr":"172.21.15.123:6801/3773700997","cluster_addr":"172.21.15.123:6803/3773700997","heartbeat_back_addr":"172.21.15.123:6807/3773700997","heartbeat_front_addr":"172.21.15.123:6805/3773700997","state":["exists","up"]},{"osd":3,"uuid":"0b364f2c-b808-4c14-b266-ae3dbe82c90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6811","nonce":2933401597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6813","nonce":2933401597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6817","nonce":2933401597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2933401597},{"type":"v1","addr":"172.21.15.67:6815","nonce":2933401597}]},"public_addr":"172.21.15.67:6811/2933401597","cluster_addr":"172.21.15.67:6813/2933401597","heartbeat_back_addr":"172.21.15.67:6817/2933401597","heartbeat_front_addr":"172.21.15.67:6815/2933401597","state":["exists","up"]},{"osd":4,"uuid":"75a0c533-a0cd-426d-9820-189b07ecef7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6809","nonce":4225579249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6811","nonce":4225579249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6815","nonce":4225579249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":4225579249},{"type":"v1","addr":"172.21.15.86:6813","nonce":4225579249}]},"public_addr":"172.21.15.86:6809/4225579249","cluster_addr":"172.21.15.86:6811/4225579249","heartbeat_back_addr":"172.21.15.86:6815/4225579249","heartbeat_front_addr":"172.21.15.86:6813/4225579249","state":["exists","up"]},{"osd":5,"uuid":"1266769f-5b0e-44be-adc4-2175cdb40312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6809","nonce":476642923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6811","nonce":476642923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6815","nonce":476642923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":476642923},{"type":"v1","addr":"172.21.15.123:6813","nonce":476642923}]},"public_addr":"172.21.15.123:6809/476642923","cluster_addr":"172.21.15.123:6811/476642923","heartbeat_back_addr":"172.21.15.123:6815/476642923","heartbeat_front_addr":"172.21.15.123:6813/476642923","state":["exists","up"]},{"osd":6,"uuid":"6bb57d51-c3a7-473e-9258-8f1dfe6c2761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6819","nonce":3799339954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6821","nonce":3799339954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6825","nonce":3799339954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3799339954},{"type":"v1","addr":"172.21.15.67:6823","nonce":3799339954}]},"public_addr":"172.21.15.67:6819/3799339954","cluster_addr":"172.21.15.67:6821/3799339954","heartbeat_back_addr":"172.21.15.67:6825/3799339954","heartbeat_front_addr":"172.21.15.67:6823/3799339954","state":["exists","up"]},{"osd":7,"uuid":"ab847d18-8863-428b-9647-eadc654dc957","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6817","nonce":3974856027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6819","nonce":3974856027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6823","nonce":3974856027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3974856027},{"type":"v1","addr":"172.21.15.86:6821","nonce":3974856027}]},"public_addr":"172.21.15.86:6817/3974856027","cluster_addr":"172.21.15.86:6819/3974856027","heartbeat_back_addr":"172.21.15.86:6823/3974856027","heartbeat_front_addr":"172.21.15.86:6821/3974856027","state":["exists","up"]},{"osd":8,"uuid":"e766f43b-63d1-4a5c-a427-2caca15c2af6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6817","nonce":1197944476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6819","nonce":1197944476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6823","nonce":1197944476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1197944476},{"type":"v1","addr":"172.21.15.123:6821","nonce":1197944476}]},"public_addr":"172.21.15.123:6817/1197944476","cluster_addr":"172.21.15.123:6819/1197944476","heartbeat_back_addr":"172.21.15.123:6823/1197944476","heartbeat_front_addr":"172.21.15.123:6821/1197944476","state":["exists","up"]},{"osd":9,"uuid":"3781cb60-d57e-40f1-a762-5ddfbfa66a15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6827","nonce":92905414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6829","nonce":92905414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6833","nonce":92905414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":92905414},{"type":"v1","addr":"172.21.15.67:6831","nonce":92905414}]},"public_addr":"172.21.15.67:6827/92905414","cluster_addr":"172.21.15.67:6829/92905414","heartbeat_back_addr":"172.21.15.67:6833/92905414","heartbeat_front_addr":"172.21.15.67:6831/92905414","state":["exists","up"]},{"osd":10,"uuid":"c315d67a-7a94-4907-9201-c76d2459b5f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6825","nonce":2965800150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6827","nonce":2965800150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6831","nonce":2965800150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2965800150},{"type":"v1","addr":"172.21.15.86:6829","nonce":2965800150}]},"public_addr":"172.21.15.86:6825/2965800150","cluster_addr":"172.21.15.86:6827/2965800150","heartbeat_back_addr":"172.21.15.86:6831/2965800150","heartbeat_front_addr":"172.21.15.86:6829/2965800150","state":["exists","up"]},{"osd":11,"uuid":"6efe153c-3d19-4e94-b8f2-40f78688c379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6825","nonce":2458295261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6827","nonce":2458295261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6831","nonce":2458295261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2458295261},{"type":"v1","addr":"172.21.15.123:6829","nonce":2458295261}]},"public_addr":"172.21.15.123:6825/2458295261","cluster_addr":"172.21.15.123:6827/2458295261","heartbeat_back_addr":"172.21.15.123:6831/2458295261","heartbeat_front_addr":"172.21.15.123:6829/2458295261","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:00.231707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:23.790471+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:10:48.296963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:14.520218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:11:39.534648+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:04.547431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:31.089528+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:12:55.964773+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:20.885143+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:13:46.742277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:11.403094+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:14:36.500997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2785096143":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3552735690":"2023-10-08T23:08:27.834910+0000","172.21.15.67:6801/2928618291":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3749023824":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1125966089":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6801/3941151123":"2023-10-08T23:08:02.345521+0000","172.21.15.67:0/1865957213":"2023-10-08T23:08:27.834910+0000","172.21.15.67:0/3750679498":"2023-10-08T23:08:02.345521+0000","172.21.15.67:6800/2928618291":"2023-10-08T23:08:27.834910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:17:31.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3cbc09b770 msgr2=0x7f3cbc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3cbc09b770 0x7f3cbc09dc20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8004600 tx=0x7f3cb800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:31.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0fefc0 msgr2=0x7f3ccc10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:31.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc10fba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00049c0 tx=0x7f3cc0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:31.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 shutdown_connections 2023-10-07T23:17:31.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0fefc0 0x7f3ccc10fba0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f3cbc09b770 0x7f3cbc09dc20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ccc100b60 0x7f3ccc117170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 --2- 172.21.15.67:0/2462822246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1001d0 0x7f3ccc1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:31.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 >> 172.21.15.67:0/2462822246 conn(0x7f3ccc0fa630 msgr2=0x7f3ccc0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:31.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 shutdown_connections 2023-10-07T23:17:31.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:31.811+0000 7f3cd1d9d700 1 -- 172.21.15.67:0/2462822246 wait complete. 2023-10-07T23:17:31.830 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T23:17:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/1869298519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0100170 msgr2=0x7fe7e0102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/1869298519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0100170 0x7fe7e0102550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40049c0 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/1869298519 shutdown_connections 2023-10-07T23:17:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/1869298519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e00fd850 0x7fe7e00ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/1869298519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e0102a90 0x7fe7e0104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/1869298519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e0100170 0x7fe7e0102550 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/1869298519 >> 172.21.15.67:0/1869298519 conn(0x7fe7e00f7370 msgr2=0x7fe7e00f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:32.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/1869298519 shutdown_connections 2023-10-07T23:17:32.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.020+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/1869298519 wait complete. 2023-10-07T23:17:32.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 Processor -- start 2023-10-07T23:17:32.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 -- start start 2023-10-07T23:17:32.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 0x7fe7e010fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e0100170 0x7fe7e01101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e0102a90 0x7fe7e01171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0117830 con 0x7fe7e00fd850 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01179a0 con 0x7fe7e0100170 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7e4f9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0117b60 con 0x7fe7e0102a90 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7df7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 0x7fe7e010fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7df7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 0x7fe7e010fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58728/0 (socket says 172.21.15.67:58728) 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7df7fe700 1 -- 172.21.15.67:0/845278801 learned_addr learned my addr 172.21.15.67:0/845278801 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.021+0000 7fe7deffd700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e0100170 0x7fe7e01101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7dffff700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e0102a90 0x7fe7e01171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7df7fe700 1 -- 172.21.15.67:0/845278801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e0102a90 msgr2=0x7fe7e01171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7df7fe700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e0102a90 0x7fe7e01171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7df7fe700 1 -- 172.21.15.67:0/845278801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e0100170 msgr2=0x7fe7e01101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7df7fe700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e0100170 0x7fe7e01101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7df7fe700 1 -- 172.21.15.67:0/845278801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d0005040 con 0x7fe7e00fd850 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7df7fe700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 0x7fe7e010fc60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009750 tx=0x7fe7d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0007c80 con 0x7fe7e00fd850 2023-10-07T23:17:32.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7d000f040 con 0x7fe7e00fd850 2023-10-07T23:17:32.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d4005040 con 0x7fe7e00fd850 2023-10-07T23:17:32.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0013c60 con 0x7fe7e00fd850 2023-10-07T23:17:32.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.022+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e0118140 con 0x7fe7e00fd850 2023-10-07T23:17:32.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.023+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e01045c0 con 0x7fe7e00fd850 2023-10-07T23:17:32.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.025+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7fe7d000e840 con 0x7fe7e00fd850 2023-10-07T23:17:32.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.026+0000 7fe7c7fff700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe7c809b770 0x7fe7c809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.027+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9103+0+0 (secure 0 0 0) 0x7fe7d00c50b0 con 0x7fe7e00fd850 2023-10-07T23:17:32.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.027+0000 7fe7deffd700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe7c809b770 0x7fe7c809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.027+0000 7fe7deffd700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe7c809b770 0x7fe7c809dc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004890 tx=0x7fe7d4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:32.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.030+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7fe7d0090510 con 0x7fe7e00fd850 2023-10-07T23:17:32.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.284+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7e002b420 con 0x7fe7e00fd850 2023-10-07T23:17:32.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.292+0000 7fe7c7fff700 1 -- 172.21.15.67:0/845278801 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7d0093b30 con 0x7fe7e00fd850 2023-10-07T23:17:32.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe7c809b770 msgr2=0x7fe7c809dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe7c809b770 0x7fe7c809dc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004890 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:32.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 msgr2=0x7fe7e010fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 0x7fe7e010fc60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009750 tx=0x7fe7d0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:32.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 shutdown_connections 2023-10-07T23:17:32.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7fe7c809b770 0x7fe7c809dc20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7e0102a90 0x7fe7e01171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7e00fd850 0x7fe7e010fc60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 --2- 172.21.15.67:0/845278801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e0100170 0x7fe7e01101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 >> 172.21.15.67:0/845278801 conn(0x7fe7e00f7370 msgr2=0x7fe7e00fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:32.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 shutdown_connections 2023-10-07T23:17:32.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.295+0000 7fe7e4f9d700 1 -- 172.21.15.67:0/845278801 wait complete. 2023-10-07T23:17:32.298 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T23:17:32.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T23:17:32.332 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[167744]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.332 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[167744]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.332 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:17:32 smithi123 ceph-mds[167768]: -- 172.21.15.123:0/2550265590 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f89b8a9c0 con 0x564f89b60c00 2023-10-07T23:17:32.332 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[167744]: starting mds.c at 2023-10-07T23:17:32.333 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[167184]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.333 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[167184]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.333 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:17:32 smithi123 ceph-mds[167208]: -- 172.21.15.123:0/3823786824 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55edacc8c9c0 con 0x55edacc62800 2023-10-07T23:17:32.333 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[167184]: starting mds.f at 2023-10-07T23:17:32.333 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[168307]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.334 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[168307]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.334 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:17:32 smithi123 ceph-mds[168331]: -- 172.21.15.123:0/3744819425 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e2cbb5a9c0 con 0x55e2cbb30800 2023-10-07T23:17:32.334 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[168307]: starting mds.i at 2023-10-07T23:17:32.334 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[166622]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.334 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[166622]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.334 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:17:32 smithi123 ceph-mds[166646]: -- 172.21.15.123:0/372765598 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b08e6429c0 con 0x55b08e619000 2023-10-07T23:17:32.335 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:17:32 smithi123 conmon[166622]: starting mds.l at 2023-10-07T23:17:32.335 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1604609275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:32.335 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:32 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2462822246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:32.335 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1604609275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:32.336 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:32 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2462822246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:32.336 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: -6101> 2023-10-07T23:15:30.978+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T23:17:32.336 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: -3456> 2023-10-07T23:15:55.981+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-07T23:17:32.336 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: -3454> 2023-10-07T23:15:55.981+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-10-07T23:17:32.336 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: -3452> 2023-10-07T23:15:55.981+0000 7f3ba31d4700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-10-07T23:17:32.337 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.337 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.337 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[166399]: starting mds.b at 2023-10-07T23:17:32.337 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[165830]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.337 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[165830]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.337 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:17:32 smithi086 ceph-mds[165854]: -- 172.21.15.86:0/3454632100 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56473ef629c0 con 0x56473ef38c00 2023-10-07T23:17:32.338 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[165830]: starting mds.h at 2023-10-07T23:17:32.338 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[164704]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.338 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[164704]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.338 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:17:32 smithi086 ceph-mds[164728]: -- 172.21.15.86:0/119648007 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562bb42e9c0 con 0x5562bb405000 2023-10-07T23:17:32.338 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[164704]: starting mds.k at 2023-10-07T23:17:32.339 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[165267]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.339 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[165267]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.339 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:17:32 smithi086 ceph-mds[165291]: -- 172.21.15.86:0/4275250380 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55920a9809c0 con 0x55920a956c00 2023-10-07T23:17:32.339 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:17:32 smithi086 conmon[165267]: starting mds.e at 2023-10-07T23:17:32.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1604609275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-07T23:17:32.340 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:32 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2462822246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-07T23:17:32.340 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[82699]: 2023-10-07T23:17:32.285+0000 7fcba4316700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:17:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 -- 172.21.15.67:0/3668500969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fccf0 msgr2=0x7f9d240fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 --2- 172.21.15.67:0/3668500969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fccf0 0x7f9d240fd100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d140049c0 tx=0x7f9d14009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 -- 172.21.15.67:0/3668500969 shutdown_connections 2023-10-07T23:17:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 --2- 172.21.15.67:0/3668500969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fe890 0x7f9d24105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 --2- 172.21.15.67:0/3668500969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fccf0 0x7f9d240fd100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 --2- 172.21.15.67:0/3668500969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d240fdf00 0x7f9d240fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 -- 172.21.15.67:0/3668500969 >> 172.21.15.67:0/3668500969 conn(0x7f9d240f8450 msgr2=0x7f9d240fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:32.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 -- 172.21.15.67:0/3668500969 shutdown_connections 2023-10-07T23:17:32.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.515+0000 7f9d2919d700 1 -- 172.21.15.67:0/3668500969 wait complete. 2023-10-07T23:17:32.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 Processor -- start 2023-10-07T23:17:32.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 -- start start 2023-10-07T23:17:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 0x7f9d2410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 0x7f9d2410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d240fe890 0x7f9d24114f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24115540 con 0x7f9d240fdf00 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241156b0 con 0x7f9d240fccf0 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2919d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24115870 con 0x7f9d240fe890 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d23fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 0x7f9d2410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d237fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 0x7f9d2410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d23fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 0x7f9d2410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.67:42406/0 (socket says 172.21.15.67:42406) 2023-10-07T23:17:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d237fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 0x7f9d2410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58788/0 (socket says 172.21.15.67:58788) 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d23fff700 1 -- 172.21.15.67:0/1356383560 learned_addr learned my addr 172.21.15.67:0/1356383560 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.516+0000 7f9d2899c700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d240fe890 0x7f9d24114f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d23fff700 1 -- 172.21.15.67:0/1356383560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d240fe890 msgr2=0x7f9d24114f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d23fff700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d240fe890 0x7f9d24114f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d23fff700 1 -- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 msgr2=0x7f9d2410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d23fff700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 0x7f9d2410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d23fff700 1 -- 172.21.15.67:0/1356383560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18005040 con 0x7f9d240fccf0 2023-10-07T23:17:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d237fe700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 0x7f9d2410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d23fff700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 0x7f9d2410d960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009b30 tx=0x7f9d14009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400e050 con 0x7f9d240fccf0 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c0048f0 con 0x7f9d240fccf0 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d1401b3f0 con 0x7f9d240fccf0 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.517+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14012650 con 0x7f9d240fccf0 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.518+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d14005040 con 0x7f9d240fccf0 2023-10-07T23:17:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.518+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d241084e0 con 0x7f9d240fccf0 2023-10-07T23:17:32.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.520+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f9d1401b560 con 0x7f9d240fccf0 2023-10-07T23:17:32.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.522+0000 7f9d20ff9700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9d0c09b770 0x7f9d0c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:32.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.522+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9533+0+0 (secure 0 0 0) 0x7f9d140c6f40 con 0x7f9d240fccf0 2023-10-07T23:17:32.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.522+0000 7f9d237fe700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9d0c09b770 0x7f9d0c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:32.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.522+0000 7f9d237fe700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9d0c09b770 0x7f9d0c09dc20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009fd0 tx=0x7f9d1800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:32.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.524+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f9d140921f0 con 0x7f9d240fccf0 2023-10-07T23:17:32.754 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[139842]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.755 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[139842]: ignoring --setgroup ceph since I am not root 2023-10-07T23:17:32.755 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:17:32 smithi067 ceph-mds[139866]: -- 172.21.15.67:0/3165254679 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a8c6429c0 con 0x559a8c618800 2023-10-07T23:17:32.755 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[139842]: starting mds.a at 2023-10-07T23:17:32.755 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[138389]: ignoring --setuser ceph since I am not root 2023-10-07T23:17:32.755 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[138389]: ignoring --setgroup ceph 2023-10-07T23:17:32.756 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[138389]: since I am not root 2023-10-07T23:17:32.756 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:17:32 smithi067 ceph-mds[138413]: -- 172.21.15.67:0/37857093 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56350da729c0 con 0x56350da48c00 2023-10-07T23:17:32.756 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:17:32 smithi067 conmon[138389]: starting mds.d at 2023-10-07T23:17:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:32.775+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9d24115f20 con 0x7f9d240fccf0 2023-10-07T23:17:33.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.292+0000 7f9d20ff9700 1 -- 172.21.15.67:0/1356383560 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9d1401f0a0 con 0x7f9d240fccf0 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9d0c09b770 msgr2=0x7f9d0c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9d0c09b770 0x7f9d0c09dc20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009fd0 tx=0x7f9d1800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 msgr2=0x7f9d2410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 0x7f9d2410d960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009b30 tx=0x7f9d14009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 shutdown_connections 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d240fccf0 0x7f9d2410d960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f9d0c09b770 0x7f9d0c09dc20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d240fe890 0x7f9d24114f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 --2- 172.21.15.67:0/1356383560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d240fdf00 0x7f9d2410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 >> 172.21.15.67:0/1356383560 conn(0x7f9d240f8450 msgr2=0x7f9d240fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 shutdown_connections 2023-10-07T23:17:33.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.296+0000 7f9d2919d700 1 -- 172.21.15.67:0/1356383560 wait complete. 2023-10-07T23:17:33.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T23:17:33.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-10-07T23:17:33.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/845278801' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: osdmap e94: 12 total, 12 up, 12 in 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/845278801' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1356383560' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:17:33.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:33 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:17:33.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-10-07T23:17:33.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/845278801' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-07T23:17:33.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-07T23:17:33.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:17:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: osdmap e94: 12 total, 12 up, 12 in 2023-10-07T23:17:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/845278801' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-07T23:17:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-07T23:17:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1356383560' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:17:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:33 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:17:33.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-10-07T23:17:33.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/845278801' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-07T23:17:33.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-07T23:17:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:17:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: osdmap e94: 12 total, 12 up, 12 in 2023-10-07T23:17:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/845278801' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-07T23:17:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-07T23:17:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1356383560' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:17:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:33 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-07T23:17:33.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 -- 172.21.15.67:0/4223259301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 msgr2=0x7feadc070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:33.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 --2- 172.21.15.67:0/4223259301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc070030 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fead00049c0 tx=0x7fead0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 -- 172.21.15.67:0/4223259301 shutdown_connections 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 --2- 172.21.15.67:0/4223259301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc104b90 0x7feadc107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 --2- 172.21.15.67:0/4223259301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc070030 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 --2- 172.21.15.67:0/4223259301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feadc070570 0x7feadc0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 -- 172.21.15.67:0/4223259301 >> 172.21.15.67:0/4223259301 conn(0x7feadc0f8410 msgr2=0x7feadc0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 -- 172.21.15.67:0/4223259301 shutdown_connections 2023-10-07T23:17:33.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.552+0000 7feae199d700 1 -- 172.21.15.67:0/4223259301 wait complete. 2023-10-07T23:17:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.553+0000 7feae199d700 1 Processor -- start 2023-10-07T23:17:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.553+0000 7feae199d700 1 -- start start 2023-10-07T23:17:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.553+0000 7feae199d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc070570 0x7feadc10fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae199d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae199d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feadc104b90 0x7feadc1171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae199d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc1177b0 con 0x7feadc071bc0 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae199d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc117920 con 0x7feadc104b90 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae199d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc117ae0 con 0x7feadc070570 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:33.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc110110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58810/0 (socket says 172.21.15.67:58810) 2023-10-07T23:17:33.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 -- 172.21.15.67:0/1145934361 learned_addr learned my addr 172.21.15.67:0/1145934361 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:33.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae099b700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc070570 0x7feadc10fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:33.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feae119c700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feadc104b90 0x7feadc1171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:33.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 -- 172.21.15.67:0/1145934361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc070570 msgr2=0x7feadc10fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:33.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc070570 0x7feadc10fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 -- 172.21.15.67:0/1145934361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feadc104b90 msgr2=0x7feadc1171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:33.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feadc104b90 0x7feadc1171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:33.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 -- 172.21.15.67:0/1145934361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0005040 con 0x7feadc071bc0 2023-10-07T23:17:33.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.554+0000 7feadbfff700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc110110 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feac800ebc0 tx=0x7feac800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:33.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.555+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac8007cc0 con 0x7feadc071bc0 2023-10-07T23:17:33.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.555+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feac800f040 con 0x7feadc071bc0 2023-10-07T23:17:33.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.555+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc117d60 con 0x7feadc071bc0 2023-10-07T23:17:33.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.555+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80094a0 con 0x7feadc071bc0 2023-10-07T23:17:33.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.555+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc118270 con 0x7feadc071bc0 2023-10-07T23:17:33.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.556+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc1032e0 con 0x7feadc071bc0 2023-10-07T23:17:33.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.558+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7feac800c600 con 0x7feadc071bc0 2023-10-07T23:17:33.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.559+0000 7fead97fa700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feacc09b770 0x7feacc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:33.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.559+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9533+0+0 (secure 0 0 0) 0x7feac80c4fc0 con 0x7feadc071bc0 2023-10-07T23:17:33.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.559+0000 7feae099b700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feacc09b770 0x7feacc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:33.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.560+0000 7feae099b700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feacc09b770 0x7feacc09dc20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fead0007f50 tx=0x7fead0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:33.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.563+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7feac804da00 con 0x7feadc071bc0 2023-10-07T23:17:33.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:33.767+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feadc10a720 con 0x7feadc071bc0 2023-10-07T23:17:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.293+0000 7fead97fa700 1 -- 172.21.15.67:0/1145934361 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7feac8019020 con 0x7feadc071bc0 2023-10-07T23:17:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feacc09b770 msgr2=0x7feacc09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feacc09b770 0x7feacc09dc20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fead0007f50 tx=0x7fead0009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 msgr2=0x7feadc110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc110110 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feac800ebc0 tx=0x7feac800b040 comp rx=0 tx=0).stop 2023-10-07T23:17:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 shutdown_connections 2023-10-07T23:17:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feadc104b90 0x7feadc1171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7feacc09b770 0x7feacc09dc20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feadc070570 0x7feadc10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 --2- 172.21.15.67:0/1145934361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feadc071bc0 0x7feadc110110 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 >> 172.21.15.67:0/1145934361 conn(0x7feadc0f8410 msgr2=0x7feadc0fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 shutdown_connections 2023-10-07T23:17:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.297+0000 7feae199d700 1 -- 172.21.15.67:0/1145934361 wait complete. 2023-10-07T23:17:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2023-10-07T23:17:34.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T23:17:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: Cluster is now healthy 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6833/432803265,v1:172.21.15.123:6834/432803265] up:boot 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6832/2419926837,v1:172.21.15.86:6835/2419926837] up:boot 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6832/1836125132,v1:172.21.15.123:6835/1836125132] up:boot 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6833/1765053211,v1:172.21.15.86:6836/1765053211] up:boot 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6834/1768645019,v1:172.21.15.86:6837/1768645019] up:boot 2023-10-07T23:17:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6836/262165249,v1:172.21.15.123:6837/262165249] up:boot 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.123:6838/185497384,v1:172.21.15.123:6839/185497384] up:boot 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.86:6838/2162083332,v1:172.21.15.86:6839/2162083332] up:boot 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6836/1241312462,v1:172.21.15.67:6837/1241312462] up:boot 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: mds.? [v2:172.21.15.67:6840/1130420785,v1:172.21.15.67:6841/1130420785] up:boot 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: fsmap 12 up:standby 2023-10-07T23:17:34.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-07T23:17:34.375 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-07T23:17:34.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-07T23:17:34.376 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:34 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1145934361' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:34.390 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: Cluster is now healthy 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6833/432803265,v1:172.21.15.123:6834/432803265] up:boot 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6832/2419926837,v1:172.21.15.86:6835/2419926837] up:boot 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6832/1836125132,v1:172.21.15.123:6835/1836125132] up:boot 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6833/1765053211,v1:172.21.15.86:6836/1765053211] up:boot 2023-10-07T23:17:34.391 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6834/1768645019,v1:172.21.15.86:6837/1768645019] up:boot 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6836/262165249,v1:172.21.15.123:6837/262165249] up:boot 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.123:6838/185497384,v1:172.21.15.123:6839/185497384] up:boot 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.86:6838/2162083332,v1:172.21.15.86:6839/2162083332] up:boot 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6836/1241312462,v1:172.21.15.67:6837/1241312462] up:boot 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: mds.? [v2:172.21.15.67:6840/1130420785,v1:172.21.15.67:6841/1130420785] up:boot 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: fsmap 12 up:standby 2023-10-07T23:17:34.392 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-07T23:17:34.393 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-07T23:17:34.394 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-07T23:17:34.394 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:34 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1145934361' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:34.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-07T23:17:34.447 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: Cluster is now healthy 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6833/432803265,v1:172.21.15.123:6834/432803265] up:boot 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6832/2419926837,v1:172.21.15.86:6835/2419926837] up:boot 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6832/1836125132,v1:172.21.15.123:6835/1836125132] up:boot 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6833/1765053211,v1:172.21.15.86:6836/1765053211] up:boot 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6834/1768645019,v1:172.21.15.86:6837/1768645019] up:boot 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6836/262165249,v1:172.21.15.123:6837/262165249] up:boot 2023-10-07T23:17:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.123:6838/185497384,v1:172.21.15.123:6839/185497384] up:boot 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.86:6838/2162083332,v1:172.21.15.86:6839/2162083332] up:boot 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6836/1241312462,v1:172.21.15.67:6837/1241312462] up:boot 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: mds.? [v2:172.21.15.67:6840/1130420785,v1:172.21.15.67:6841/1130420785] up:boot 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: fsmap 12 up:standby 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-07T23:17:34.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-07T23:17:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='mgr.14152 172.21.15.67:0/2603288923' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-07T23:17:34.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:34 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1145934361' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 -- 172.21.15.67:0/462937110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fe890 msgr2=0x7f1e5c105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/462937110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fe890 0x7f1e5c105150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009d60 tx=0x7f1e54004d10 comp rx=0 tx=0).stop 2023-10-07T23:17:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 -- 172.21.15.67:0/462937110 shutdown_connections 2023-10-07T23:17:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/462937110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5c0fccf0 0x7f1e5c0fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/462937110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fe890 0x7f1e5c105150 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/462937110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c0fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 -- 172.21.15.67:0/462937110 >> 172.21.15.67:0/462937110 conn(0x7f1e5c0f8410 msgr2=0x7f1e5c0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 -- 172.21.15.67:0/462937110 shutdown_connections 2023-10-07T23:17:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 -- 172.21.15.67:0/462937110 wait complete. 2023-10-07T23:17:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.518+0000 7f1e5b59e700 1 Processor -- start 2023-10-07T23:17:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 -- start start 2023-10-07T23:17:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5c0fccf0 0x7f1e5c10d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c10dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e5c0fe890 0x7f1e5c114d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c115320 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c115490 con 0x7f1e5c0fccf0 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c115650 con 0x7f1e5c0fe890 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e59d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c10dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e59d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c10dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58830/0 (socket says 172.21.15.67:58830) 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e59d9b700 1 -- 172.21.15.67:0/2914435184 learned_addr learned my addr 172.21.15.67:0/2914435184 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5a59c700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5c0fccf0 0x7f1e5c10d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.519+0000 7f1e5ad9d700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e5c0fe890 0x7f1e5c114d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e59d9b700 1 -- 172.21.15.67:0/2914435184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e5c0fe890 msgr2=0x7f1e5c114d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e59d9b700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e5c0fe890 0x7f1e5c114d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e59d9b700 1 -- 172.21.15.67:0/2914435184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5c0fccf0 msgr2=0x7f1e5c10d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e59d9b700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5c0fccf0 0x7f1e5c10d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e59d9b700 1 -- 172.21.15.67:0/2914435184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54009940 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e59d9b700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c10dc80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000ebf0 tx=0x7f1e5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500047d0 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c1158d0 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e5000f040 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000c980 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.520+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c115d70 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.521+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c108510 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.523+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f1e500090d0 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.525+0000 7f1e4affd700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1e4409b770 0x7f1e4409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.525+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8950+0+0 (secure 0 0 0) 0x7f1e500c4e30 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.525+0000 7f1e5a59c700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1e4409b770 0x7f1e4409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.525+0000 7f1e5a59c700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1e4409b770 0x7f1e4409dc20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009fd0 tx=0x7f1e4c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:34.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.528+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f1e50090320 con 0x7f1e5c0fdf00 2023-10-07T23:17:34.732 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:34.732+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e5c103920 con 0x7f1e5c0fdf00 2023-10-07T23:17:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.301+0000 7f1e4affd700 1 -- 172.21.15.67:0/2914435184 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1e50019020 con 0x7f1e5c0fdf00 2023-10-07T23:17:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1e4409b770 msgr2=0x7f1e4409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1e4409b770 0x7f1e4409dc20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009fd0 tx=0x7f1e4c00b040 comp rx=0 tx=0).stop 2023-10-07T23:17:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 msgr2=0x7f1e5c10dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c10dc80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000ebf0 tx=0x7f1e5000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 shutdown_connections 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e5c0fccf0 0x7f1e5c10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f1e4409b770 0x7f1e4409dc20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e5c0fe890 0x7f1e5c114d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 --2- 172.21.15.67:0/2914435184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e5c0fdf00 0x7f1e5c10dc80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 >> 172.21.15.67:0/2914435184 conn(0x7f1e5c0f8410 msgr2=0x7f1e5c0fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 shutdown_connections 2023-10-07T23:17:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.304+0000 7f1e5b59e700 1 -- 172.21.15.67:0/2914435184 wait complete. 2023-10-07T23:17:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2023-10-07T23:17:35.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-07T23:17:35.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.522+0000 7f8469f9d700 1 -- 172.21.15.67:0/901680566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 msgr2=0x7f84640fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.522+0000 7f8469f9d700 1 --2- 172.21.15.67:0/901680566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f84640fd170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f84580049c0 tx=0x7f8458009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 -- 172.21.15.67:0/901680566 shutdown_connections 2023-10-07T23:17:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 --2- 172.21.15.67:0/901680566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84640fe900 0x7f84641051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 --2- 172.21.15.67:0/901680566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f84640fd170 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 --2- 172.21.15.67:0/901680566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fdf70 0x7f84640fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 -- 172.21.15.67:0/901680566 >> 172.21.15.67:0/901680566 conn(0x7f84640f84c0 msgr2=0x7f84640fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 -- 172.21.15.67:0/901680566 shutdown_connections 2023-10-07T23:17:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 -- 172.21.15.67:0/901680566 wait complete. 2023-10-07T23:17:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.523+0000 7f8469f9d700 1 Processor -- start 2023-10-07T23:17:35.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 -- start start 2023-10-07T23:17:35.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f8464071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:35.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84640fdf70 0x7f846406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:35.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fe900 0x7f84640704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464070c40 con 0x7f84640fcd60 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464070db0 con 0x7f84640fdf70 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8469f9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464117ae0 con 0x7f84640fe900 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8468f9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f8464071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8468f9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f8464071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58850/0 (socket says 172.21.15.67:58850) 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8468f9b700 1 -- 172.21.15.67:0/1742475349 learned_addr learned my addr 172.21.15.67:0/1742475349 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-07T23:17:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f8463fff700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84640fdf70 0x7f846406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.524+0000 7f846979c700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fe900 0x7f84640704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8468f9b700 1 -- 172.21.15.67:0/1742475349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fe900 msgr2=0x7f84640704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8468f9b700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fe900 0x7f84640704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8468f9b700 1 -- 172.21.15.67:0/1742475349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84640fdf70 msgr2=0x7f846406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8468f9b700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84640fdf70 0x7f846406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8468f9b700 1 -- 172.21.15.67:0/1742475349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458005040 con 0x7f84640fcd60 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f846979c700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fe900 0x7f84640704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:17:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8468f9b700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f8464071930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8458009040 tx=0x7f8458009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:35.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800e040 con 0x7f84640fcd60 2023-10-07T23:17:35.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8458009670 con 0x7f84640fcd60 2023-10-07T23:17:35.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8464117d30 con 0x7f84640fcd60 2023-10-07T23:17:35.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458013df0 con 0x7f84640fcd60 2023-10-07T23:17:35.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.525+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8464118190 con 0x7f84640fcd60 2023-10-07T23:17:35.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.526+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8464104330 con 0x7f84640fcd60 2023-10-07T23:17:35.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.528+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 129339+0+0 (secure 0 0 0) 0x7f84580091a0 con 0x7f84640fcd60 2023-10-07T23:17:35.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.529+0000 7f84617fa700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f845409b770 0x7f845409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:17:35.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.530+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8420+0+0 (secure 0 0 0) 0x7f84580c6360 con 0x7f84640fcd60 2023-10-07T23:17:35.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.530+0000 7f8463fff700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f845409b770 0x7f845409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:17:35.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.530+0000 7f8463fff700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f845409b770 0x7f845409dc20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8450009b80 tx=0x7f845000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:17:35.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.533+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174570 (secure 0 0 0) 0x7f8458091a60 con 0x7f84640fcd60 2023-10-07T23:17:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:35 smithi123 ceph-mon[122926]: pgmap v357: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-07T23:17:35.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:35 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1145934361' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-07T23:17:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:35 smithi123 ceph-mon[122926]: osdmap e95: 12 total, 12 up, 12 in 2023-10-07T23:17:35.641 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:35 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2914435184' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:35 smithi086 ceph-mon[122131]: pgmap v357: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-07T23:17:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:35 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1145934361' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-07T23:17:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:35 smithi086 ceph-mon[122131]: osdmap e95: 12 total, 12 up, 12 in 2023-10-07T23:17:35.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:35 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2914435184' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:35.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:35.741+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84641084b0 con 0x7f84640fcd60 2023-10-07T23:17:35.754 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:35 smithi067 ceph-mon[82723]: pgmap v357: 129 pgs: 129 active+clean; 586 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-07T23:17:35.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:35 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1145934361' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-07T23:17:35.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:35 smithi067 ceph-mon[82723]: osdmap e95: 12 total, 12 up, 12 in 2023-10-07T23:17:35.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:35 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2914435184' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:36.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.312+0000 7f84617fa700 1 -- 172.21.15.67:0/1742475349 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8458095080 con 0x7f84640fcd60 2023-10-07T23:17:36.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f845409b770 msgr2=0x7f845409dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:36.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f845409b770 0x7f845409dc20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8450009b80 tx=0x7f845000b040 comp rx=0 tx=0).stop 2023-10-07T23:17:36.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 msgr2=0x7f8464071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:17:36.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f8464071930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8458009040 tx=0x7f8458009e30 comp rx=0 tx=0).stop 2023-10-07T23:17:36.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 shutdown_connections 2023-10-07T23:17:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:6800/2299203628,v1:172.21.15.67:6801/2299203628] conn(0x7f845409b770 0x7f845409dc20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84640fe900 0x7f84640704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84640fcd60 0x7f8464071930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 --2- 172.21.15.67:0/1742475349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84640fdf70 0x7f846406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:17:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.316+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 >> 172.21.15.67:0/1742475349 conn(0x7f84640f84c0 msgr2=0x7f84641014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:17:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.317+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 shutdown_connections 2023-10-07T23:17:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-07T23:17:36.317+0000 7f8469f9d700 1 -- 172.21.15.67:0/1742475349 wait complete. 2023-10-07T23:17:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' removed 2023-10-07T23:17:36.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:17:36.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:17:36.449 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:17:36.494 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-07T23:17:36.495 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-07T23:17:36.531 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-07T23:17:36.547 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-07T23:17:36.563 INFO:tasks.cephadm:Stopping all daemons... 2023-10-07T23:17:36.563 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-07T23:17:36.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a 2023-10-07T23:17:36.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:36 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/2914435184' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-07T23:17:36.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:36 smithi067 ceph-mon[82723]: osdmap e96: 12 total, 12 up, 12 in 2023-10-07T23:17:36.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:36 smithi067 ceph-mon[82723]: from='client.? 172.21.15.67:0/1742475349' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:36.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:36 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/2914435184' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-07T23:17:36.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:36 smithi123 ceph-mon[122926]: osdmap e96: 12 total, 12 up, 12 in 2023-10-07T23:17:36.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:36 smithi123 ceph-mon[122926]: from='client.? 172.21.15.67:0/1742475349' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:36.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:36 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/2914435184' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-07T23:17:36.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:36 smithi086 ceph-mon[122131]: osdmap e96: 12 total, 12 up, 12 in 2023-10-07T23:17:36.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:36 smithi086 ceph-mon[122131]: from='client.? 172.21.15.67:0/1742475349' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-07T23:17:37.005 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:36 smithi067 systemd[1]: Stopping Ceph mon.a for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:37.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:37 smithi067 conmon[82699]: 2023-10-07T23:17:37.085+0000 7fcba9b21700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:17:37.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:37 smithi067 conmon[82699]: 2023-10-07T23:17:37.085+0000 7fcba9b21700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-07T23:17:37.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Oct 07 23:17:37 smithi067 bash[162930]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mon-a 2023-10-07T23:17:37.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-07T23:17:37.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:37.895 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-07T23:17:37.895 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-07T23:17:37.896 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.b 2023-10-07T23:17:38.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:37 smithi086 systemd[1]: Stopping Ceph mon.b for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:38.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:38 smithi086 conmon[122106]: 2023-10-07T23:17:38.290+0000 7f13c9b93700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:17:38.697 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:38 smithi086 conmon[122106]: 2023-10-07T23:17:38.290+0000 7f13c9b93700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-07T23:17:39.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-07T23:17:39.263 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:38 smithi086 bash[169244]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mon-b 2023-10-07T23:17:39.264 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:39 smithi086 systemd[1]: ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.b.service: Succeeded. 2023-10-07T23:17:39.264 INFO:journalctl@ceph.mon.b.smithi086.stdout:Oct 07 23:17:39 smithi086 systemd[1]: Stopped Ceph mon.b for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:17:39.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:39.300 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-07T23:17:39.300 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-07T23:17:39.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.c 2023-10-07T23:17:39.640 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:39 smithi123 systemd[1]: Stopping Ceph mon.c for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:40.004 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:39 smithi123 conmon[122893]: 2023-10-07T23:17:39.650+0000 7f176e235700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:17:40.004 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:39 smithi123 conmon[122893]: 2023-10-07T23:17:39.650+0000 7f176e235700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-07T23:17:40.005 INFO:journalctl@ceph.mon.c.smithi123.stdout:Oct 07 23:17:39 smithi123 bash[171480]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mon-c 2023-10-07T23:17:40.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-07T23:17:40.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:40.702 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-07T23:17:40.702 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-07T23:17:40.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x 2023-10-07T23:17:41.219 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:17:40 smithi067 systemd[1]: Stopping Ceph mgr.x for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:41.471 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:17:41 smithi067 conmon[83139]: 2023-10-07T23:17:41.217+0000 7f854b633700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:17:41.471 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Oct 07 23:17:41 smithi067 bash[163939]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mgr-x 2023-10-07T23:17:41.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-07T23:17:41.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:41.910 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-07T23:17:41.911 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-07T23:17:41.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.y 2023-10-07T23:17:42.198 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:17:41 smithi086 systemd[1]: Stopping Ceph mgr.y for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:43.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-07T23:17:43.182 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:17:42 smithi086 bash[169938]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mgr-y 2023-10-07T23:17:43.182 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:17:42 smithi086 systemd[1]: ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-07T23:17:43.183 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:17:43 smithi086 systemd[1]: ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.y.service: Failed with result 'exit-code'. 2023-10-07T23:17:43.183 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Oct 07 23:17:43 smithi086 systemd[1]: Stopped Ceph mgr.y for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:17:43.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:43.196 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-07T23:17:43.196 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-07T23:17:43.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.z 2023-10-07T23:17:43.640 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Oct 07 23:17:43 smithi123 systemd[1]: Stopping Ceph mgr.z for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:44.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-07T23:17:44.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:44.377 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-07T23:17:44.378 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-07T23:17:44.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.0 2023-10-07T23:17:44.754 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:17:44 smithi067 systemd[1]: Stopping Ceph osd.0 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:45.254 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:17:44 smithi067 conmon[101320]: 2023-10-07T23:17:44.893+0000 7fced82fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:17:45.254 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:17:44 smithi067 conmon[101320]: 2023-10-07T23:17:44.893+0000 7fced82fb700 -1 osd.0 97 *** Got signal Terminated *** 2023-10-07T23:17:45.255 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:17:44 smithi067 conmon[101320]: 2023-10-07T23:17:44.893+0000 7fced82fb700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:17:51.004 INFO:journalctl@ceph.osd.0.smithi067.stdout:Oct 07 23:17:50 smithi067 bash[164230]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-0 2023-10-07T23:17:52.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-07T23:17:52.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:17:52.482 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-07T23:17:52.482 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-07T23:17:52.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.1 2023-10-07T23:17:52.834 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:17:52 smithi086 systemd[1]: Stopping Ceph osd.1 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:17:53.197 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:17:52 smithi086 conmon[128550]: 2023-10-07T23:17:52.832+0000 7f82e4fc9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:17:53.197 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:17:52 smithi086 conmon[128550]: 2023-10-07T23:17:52.832+0000 7f82e4fc9700 -1 osd.1 97 *** Got signal Terminated *** 2023-10-07T23:17:53.197 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:17:52 smithi086 conmon[128550]: 2023-10-07T23:17:52.832+0000 7f82e4fc9700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:17:58.697 INFO:journalctl@ceph.osd.1.smithi086.stdout:Oct 07 23:17:58 smithi086 bash[170219]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-1 2023-10-07T23:18:00.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-07T23:18:00.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:00.566 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-07T23:18:00.566 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-07T23:18:00.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.2 2023-10-07T23:18:00.890 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:18:00 smithi123 systemd[1]: Stopping Ceph osd.2 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:01.390 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:18:00 smithi123 conmon[130600]: 2023-10-07T23:18:00.917+0000 7fddeb566700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:01.391 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:18:00 smithi123 conmon[130600]: 2023-10-07T23:18:00.917+0000 7fddeb566700 -1 osd.2 97 *** Got signal Terminated *** 2023-10-07T23:18:01.391 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:18:00 smithi123 conmon[130600]: 2023-10-07T23:18:00.917+0000 7fddeb566700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:07.390 INFO:journalctl@ceph.osd.2.smithi123.stdout:Oct 07 23:18:07 smithi123 bash[172133]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-2 2023-10-07T23:18:08.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-07T23:18:08.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:08.860 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-07T23:18:08.860 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-07T23:18:08.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.3 2023-10-07T23:18:09.254 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:08 smithi067 systemd[1]: Stopping Ceph osd.3 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:09.754 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:09 smithi067 conmon[111399]: 2023-10-07T23:18:09.372+0000 7f4a0f3c8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:09.754 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:09 smithi067 conmon[111399]: 2023-10-07T23:18:09.372+0000 7f4a0f3c8700 -1 osd.3 97 *** Got signal Terminated *** 2023-10-07T23:18:09.755 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:09 smithi067 conmon[111399]: 2023-10-07T23:18:09.372+0000 7f4a0f3c8700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:12.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:12 smithi123 conmon[160185]: 2023-10-07T23:18:12.027+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:12.504 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:12 smithi067 conmon[111399]: 2023-10-07T23:18:12.210+0000 7f4a0b9c6700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:45.590867+0000 front 2023-10-07T23:17:45.590934+0000 (oldest deadline 2023-10-07T23:18:11.491018+0000) 2023-10-07T23:18:13.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:12 smithi067 conmon[130383]: 2023-10-07T23:18:12.700+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:13.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:13 smithi123 conmon[160185]: 2023-10-07T23:18:13.058+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:13.504 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:13 smithi067 conmon[111399]: 2023-10-07T23:18:13.172+0000 7f4a0b9c6700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:45.590867+0000 front 2023-10-07T23:17:45.590934+0000 (oldest deadline 2023-10-07T23:18:11.491018+0000) 2023-10-07T23:18:14.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:13 smithi067 conmon[130383]: 2023-10-07T23:18:13.653+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:14.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:14 smithi123 conmon[160185]: 2023-10-07T23:18:14.013+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:14.447 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:14 smithi086 conmon[158442]: 2023-10-07T23:18:14.040+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:14.504 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:14 smithi067 conmon[111399]: 2023-10-07T23:18:14.179+0000 7f4a0b9c6700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:45.590867+0000 front 2023-10-07T23:17:45.590934+0000 (oldest deadline 2023-10-07T23:18:11.491018+0000) 2023-10-07T23:18:14.930 INFO:journalctl@ceph.osd.3.smithi067.stdout:Oct 07 23:18:14 smithi067 bash[164785]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-3 2023-10-07T23:18:14.931 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:14 smithi067 conmon[130383]: 2023-10-07T23:18:14.657+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:15.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:14 smithi123 conmon[160185]: 2023-10-07T23:18:14.976+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:15.447 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:14 smithi086 conmon[158442]: 2023-10-07T23:18:14.990+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:15.890 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:15 smithi123 conmon[140237]: 2023-10-07T23:18:15.484+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:15.952 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:15 smithi067 conmon[130383]: 2023-10-07T23:18:15.677+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:16.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:15 smithi123 conmon[160185]: 2023-10-07T23:18:15.929+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:16.447 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:15 smithi086 conmon[158442]: 2023-10-07T23:18:15.978+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:16.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-07T23:18:16.890 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:16 smithi123 conmon[140237]: 2023-10-07T23:18:16.511+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:16.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:16.951 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-07T23:18:16.951 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-07T23:18:16.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.4 2023-10-07T23:18:17.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:16 smithi067 conmon[130383]: 2023-10-07T23:18:16.726+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:17.313 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:16 smithi086 systemd[1]: Stopping Ceph osd.4 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:17.314 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:16 smithi086 conmon[158442]: 2023-10-07T23:18:16.980+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:17.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:16 smithi123 conmon[160185]: 2023-10-07T23:18:16.908+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:17.697 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:17 smithi086 conmon[138642]: 2023-10-07T23:18:17.313+0000 7fd7106d7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:17.697 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:17 smithi086 conmon[138642]: 2023-10-07T23:18:17.313+0000 7fd7106d7700 -1 osd.4 97 *** Got signal Terminated *** 2023-10-07T23:18:17.697 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:17 smithi086 conmon[138642]: 2023-10-07T23:18:17.313+0000 7fd7106d7700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:17.890 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:17 smithi123 conmon[140237]: 2023-10-07T23:18:17.473+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:18.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:17 smithi067 conmon[130383]: 2023-10-07T23:18:17.679+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:18.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:17 smithi086 conmon[158442]: 2023-10-07T23:18:17.931+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:18.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:17 smithi123 conmon[160185]: 2023-10-07T23:18:17.905+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:18.878 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:18 smithi123 conmon[140237]: 2023-10-07T23:18:18.428+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:19.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:18 smithi067 conmon[130383]: 2023-10-07T23:18:18.681+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:19.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:18 smithi123 conmon[160185]: 2023-10-07T23:18:18.877+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:19.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:18 smithi086 conmon[158442]: 2023-10-07T23:18:18.895+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:19.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:18 smithi086 conmon[158442]: 2023-10-07T23:18:18.895+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:19.692 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:19 smithi067 conmon[120945]: 2023-10-07T23:18:19.308+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:19.890 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:19 smithi123 conmon[140237]: 2023-10-07T23:18:19.474+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:19.891 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:19 smithi123 conmon[140237]: 2023-10-07T23:18:19.474+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:20.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:19 smithi067 conmon[130383]: 2023-10-07T23:18:19.692+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:20.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:19 smithi086 conmon[158442]: 2023-10-07T23:18:19.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:20.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:19 smithi086 conmon[158442]: 2023-10-07T23:18:19.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:20.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:19 smithi123 conmon[160185]: 2023-10-07T23:18:19.898+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:20.734 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:20 smithi067 conmon[120945]: 2023-10-07T23:18:20.262+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:20.863 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:20 smithi123 conmon[140237]: 2023-10-07T23:18:20.450+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:20.864 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:20 smithi123 conmon[140237]: 2023-10-07T23:18:20.450+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:20.947 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:20 smithi086 conmon[138642]: 2023-10-07T23:18:20.637+0000 7fd709ccf700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.470294+0000 front 2023-10-07T23:17:56.470314+0000 (oldest deadline 2023-10-07T23:18:19.970573+0000) 2023-10-07T23:18:20.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:20 smithi086 conmon[158442]: 2023-10-07T23:18:20.853+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:20.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:20 smithi086 conmon[158442]: 2023-10-07T23:18:20.853+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:21.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:20 smithi067 conmon[130383]: 2023-10-07T23:18:20.733+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:21.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:20 smithi123 conmon[160185]: 2023-10-07T23:18:20.863+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:21.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:21 smithi123 conmon[150509]: 2023-10-07T23:18:21.381+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:21.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:21 smithi123 conmon[140237]: 2023-10-07T23:18:21.431+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:21.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:21 smithi123 conmon[140237]: 2023-10-07T23:18:21.431+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:21.736 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:21 smithi067 conmon[120945]: 2023-10-07T23:18:21.288+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:21.947 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:21 smithi086 conmon[138642]: 2023-10-07T23:18:21.611+0000 7fd709ccf700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.470294+0000 front 2023-10-07T23:17:56.470314+0000 (oldest deadline 2023-10-07T23:18:19.970573+0000) 2023-10-07T23:18:21.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:21 smithi086 conmon[158442]: 2023-10-07T23:18:21.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:21.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:21 smithi086 conmon[158442]: 2023-10-07T23:18:21.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:22.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:21 smithi067 conmon[130383]: 2023-10-07T23:18:21.735+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:22.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:21 smithi123 conmon[160185]: 2023-10-07T23:18:21.823+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:22.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:22 smithi067 conmon[120945]: 2023-10-07T23:18:22.247+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:22.640 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:22 smithi123 conmon[140237]: 2023-10-07T23:18:22.386+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:22.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:22 smithi123 conmon[140237]: 2023-10-07T23:18:22.386+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:22.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:22 smithi123 conmon[150509]: 2023-10-07T23:18:22.388+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:23.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:22 smithi067 conmon[130383]: 2023-10-07T23:18:22.716+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:23.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:22 smithi067 conmon[130383]: 2023-10-07T23:18:22.716+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:23.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:22 smithi123 conmon[160185]: 2023-10-07T23:18:22.828+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:23.197 INFO:journalctl@ceph.osd.4.smithi086.stdout:Oct 07 23:18:22 smithi086 bash[170769]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-4 2023-10-07T23:18:23.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:23 smithi086 conmon[148156]: 2023-10-07T23:18:23.017+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:23.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:22 smithi086 conmon[158442]: 2023-10-07T23:18:22.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:23.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:22 smithi086 conmon[158442]: 2023-10-07T23:18:22.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:23.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:23 smithi123 conmon[150509]: 2023-10-07T23:18:23.365+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:23.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:23 smithi123 conmon[140237]: 2023-10-07T23:18:23.426+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:23.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:23 smithi123 conmon[140237]: 2023-10-07T23:18:23.426+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:23.678 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:23 smithi067 conmon[120945]: 2023-10-07T23:18:23.274+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:24.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:23 smithi067 conmon[130383]: 2023-10-07T23:18:23.678+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:24.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:23 smithi067 conmon[130383]: 2023-10-07T23:18:23.678+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:24.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:23 smithi123 conmon[160185]: 2023-10-07T23:18:23.838+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:24.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:23 smithi123 conmon[160185]: 2023-10-07T23:18:23.838+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:24.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:24 smithi086 conmon[148156]: 2023-10-07T23:18:24.040+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:24.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:23 smithi086 conmon[158442]: 2023-10-07T23:18:23.843+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:24.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:23 smithi086 conmon[158442]: 2023-10-07T23:18:23.843+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:24.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:24 smithi067 conmon[120945]: 2023-10-07T23:18:24.251+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:24.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:24 smithi123 conmon[150509]: 2023-10-07T23:18:24.333+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:24.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:24 smithi123 conmon[140237]: 2023-10-07T23:18:24.467+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:24.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:24 smithi123 conmon[140237]: 2023-10-07T23:18:24.467+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:24.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-07T23:18:24.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:24.980 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-07T23:18:24.980 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-07T23:18:24.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.5 2023-10-07T23:18:25.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:24 smithi067 conmon[130383]: 2023-10-07T23:18:24.670+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:25.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:24 smithi067 conmon[130383]: 2023-10-07T23:18:24.670+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:25.140 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:25 smithi123 systemd[1]: Stopping Ceph osd.5 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:25.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:24 smithi123 conmon[160185]: 2023-10-07T23:18:24.834+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:25.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:24 smithi123 conmon[160185]: 2023-10-07T23:18:24.834+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:25.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:25 smithi086 conmon[148156]: 2023-10-07T23:18:25.057+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:25.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:24 smithi086 conmon[158442]: 2023-10-07T23:18:24.875+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:25.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:24 smithi086 conmon[158442]: 2023-10-07T23:18:24.875+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:25.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[150509]: 2023-10-07T23:18:25.365+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:25.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[140237]: 2023-10-07T23:18:25.331+0000 7f3a4e9c7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:25.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[140237]: 2023-10-07T23:18:25.331+0000 7f3a4e9c7700 -1 osd.5 97 *** Got signal Terminated *** 2023-10-07T23:18:25.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[140237]: 2023-10-07T23:18:25.331+0000 7f3a4e9c7700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:25.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[140237]: 2023-10-07T23:18:25.487+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:25.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[140237]: 2023-10-07T23:18:25.487+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:25.701 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:25 smithi067 conmon[120945]: 2023-10-07T23:18:25.294+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:26.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:25 smithi067 conmon[130383]: 2023-10-07T23:18:25.701+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:26.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:25 smithi067 conmon[130383]: 2023-10-07T23:18:25.701+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:26.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[160185]: 2023-10-07T23:18:25.818+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:26.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:25 smithi123 conmon[160185]: 2023-10-07T23:18:25.818+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:26.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:26 smithi086 conmon[148156]: 2023-10-07T23:18:26.043+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:26.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:25 smithi086 conmon[158442]: 2023-10-07T23:18:25.831+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:26.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:25 smithi086 conmon[158442]: 2023-10-07T23:18:25.831+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:26.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:26 smithi123 conmon[150509]: 2023-10-07T23:18:26.330+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:26.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:26 smithi123 conmon[140237]: 2023-10-07T23:18:26.439+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:26.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:26 smithi123 conmon[140237]: 2023-10-07T23:18:26.439+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:26.731 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:26 smithi067 conmon[120945]: 2023-10-07T23:18:26.263+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:27.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:26 smithi067 conmon[130383]: 2023-10-07T23:18:26.730+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:27.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:26 smithi067 conmon[130383]: 2023-10-07T23:18:26.730+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:27.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:26 smithi123 conmon[160185]: 2023-10-07T23:18:26.857+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:27.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:26 smithi123 conmon[160185]: 2023-10-07T23:18:26.857+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:27.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:27 smithi086 conmon[148156]: 2023-10-07T23:18:27.086+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:27.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:26 smithi086 conmon[158442]: 2023-10-07T23:18:26.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:27.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:26 smithi086 conmon[158442]: 2023-10-07T23:18:26.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:27.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:26 smithi086 conmon[158442]: 2023-10-07T23:18:26.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:27.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:27 smithi067 conmon[120945]: 2023-10-07T23:18:27.231+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:27.640 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:27 smithi123 conmon[140237]: 2023-10-07T23:18:27.398+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:27.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:27 smithi123 conmon[140237]: 2023-10-07T23:18:27.398+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:27.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:27 smithi123 conmon[150509]: 2023-10-07T23:18:27.349+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:27.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:27 smithi123 conmon[150509]: 2023-10-07T23:18:27.349+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:28.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:27 smithi067 conmon[130383]: 2023-10-07T23:18:27.688+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:28.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:27 smithi067 conmon[130383]: 2023-10-07T23:18:27.688+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:28.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:27 smithi123 conmon[160185]: 2023-10-07T23:18:27.811+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:28.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:27 smithi123 conmon[160185]: 2023-10-07T23:18:27.811+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:28.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:28 smithi086 conmon[148156]: 2023-10-07T23:18:28.079+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:28.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:27 smithi086 conmon[158442]: 2023-10-07T23:18:27.841+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:28.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:27 smithi086 conmon[158442]: 2023-10-07T23:18:27.841+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:28.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:27 smithi086 conmon[158442]: 2023-10-07T23:18:27.841+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:28.640 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[140237]: 2023-10-07T23:18:28.365+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:28.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[140237]: 2023-10-07T23:18:28.365+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:28.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[150509]: 2023-10-07T23:18:28.321+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:28.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[150509]: 2023-10-07T23:18:28.321+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:28.709 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:28 smithi067 conmon[120945]: 2023-10-07T23:18:28.281+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:29.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:28 smithi067 conmon[130383]: 2023-10-07T23:18:28.709+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:29.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:28 smithi067 conmon[130383]: 2023-10-07T23:18:28.709+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:29.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[160185]: 2023-10-07T23:18:28.796+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:29.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[160185]: 2023-10-07T23:18:28.796+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:29.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:28 smithi123 conmon[160185]: 2023-10-07T23:18:28.796+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:29.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:29 smithi086 conmon[148156]: 2023-10-07T23:18:29.089+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:29.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:29 smithi086 conmon[148156]: 2023-10-07T23:18:29.089+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:29.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:28 smithi086 conmon[158442]: 2023-10-07T23:18:28.865+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:29.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:28 smithi086 conmon[158442]: 2023-10-07T23:18:28.865+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:29.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:28 smithi086 conmon[158442]: 2023-10-07T23:18:28.865+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:29.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:29 smithi067 conmon[120945]: 2023-10-07T23:18:29.240+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:29.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:29 smithi067 conmon[120945]: 2023-10-07T23:18:29.240+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:29.640 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[140237]: 2023-10-07T23:18:29.407+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:47.651950+0000 front 2023-10-07T23:17:47.651837+0000 (oldest deadline 2023-10-07T23:18:15.252272+0000) 2023-10-07T23:18:29.640 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[140237]: 2023-10-07T23:18:29.407+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.252600+0000 front 2023-10-07T23:17:55.252579+0000 (oldest deadline 2023-10-07T23:18:19.352721+0000) 2023-10-07T23:18:29.641 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[140237]: 2023-10-07T23:18:29.407+0000 7f3a4bfc7700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.954345+0000 front 2023-10-07T23:18:03.954324+0000 (oldest deadline 2023-10-07T23:18:28.654422+0000) 2023-10-07T23:18:29.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[150509]: 2023-10-07T23:18:29.275+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:29.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[150509]: 2023-10-07T23:18:29.275+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:30.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:29 smithi067 conmon[130383]: 2023-10-07T23:18:29.682+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:30.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:29 smithi067 conmon[130383]: 2023-10-07T23:18:29.682+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:30.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:29 smithi067 conmon[130383]: 2023-10-07T23:18:29.682+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:30.116 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:29 smithi086 conmon[158442]: 2023-10-07T23:18:29.851+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:30.116 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:29 smithi086 conmon[158442]: 2023-10-07T23:18:29.851+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:30.117 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:29 smithi086 conmon[158442]: 2023-10-07T23:18:29.851+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:30.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[160185]: 2023-10-07T23:18:29.829+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:30.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[160185]: 2023-10-07T23:18:29.829+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:30.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:29 smithi123 conmon[160185]: 2023-10-07T23:18:29.829+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:30.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:30 smithi086 conmon[148156]: 2023-10-07T23:18:30.116+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:30.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:30 smithi086 conmon[148156]: 2023-10-07T23:18:30.116+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:30.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:30 smithi067 conmon[120945]: 2023-10-07T23:18:30.253+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:30.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:30 smithi067 conmon[120945]: 2023-10-07T23:18:30.253+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:30.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:30 smithi123 conmon[150509]: 2023-10-07T23:18:30.284+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:30.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:30 smithi123 conmon[150509]: 2023-10-07T23:18:30.284+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:31.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:30 smithi067 conmon[130383]: 2023-10-07T23:18:30.679+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:31.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:30 smithi067 conmon[130383]: 2023-10-07T23:18:30.679+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:31.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:30 smithi067 conmon[130383]: 2023-10-07T23:18:30.679+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:31.096 INFO:journalctl@ceph.osd.5.smithi123.stdout:Oct 07 23:18:30 smithi123 bash[172675]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-5 2023-10-07T23:18:31.096 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:30 smithi123 conmon[160185]: 2023-10-07T23:18:30.811+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:31.096 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:30 smithi123 conmon[160185]: 2023-10-07T23:18:30.811+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:31.097 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:30 smithi123 conmon[160185]: 2023-10-07T23:18:30.811+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:31.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:31 smithi086 conmon[148156]: 2023-10-07T23:18:31.127+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:31.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:31 smithi086 conmon[148156]: 2023-10-07T23:18:31.127+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:31.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:30 smithi086 conmon[158442]: 2023-10-07T23:18:30.888+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:31.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:30 smithi086 conmon[158442]: 2023-10-07T23:18:30.888+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:31.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:30 smithi086 conmon[158442]: 2023-10-07T23:18:30.888+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:31.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:31 smithi123 conmon[150509]: 2023-10-07T23:18:31.309+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:31.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:31 smithi123 conmon[150509]: 2023-10-07T23:18:31.309+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:31.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:31 smithi067 conmon[120945]: 2023-10-07T23:18:31.207+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:31.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:31 smithi067 conmon[120945]: 2023-10-07T23:18:31.207+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:32.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:31 smithi067 conmon[130383]: 2023-10-07T23:18:31.712+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:32.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:31 smithi067 conmon[130383]: 2023-10-07T23:18:31.712+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:32.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:31 smithi067 conmon[130383]: 2023-10-07T23:18:31.712+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:32.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:31 smithi123 conmon[160185]: 2023-10-07T23:18:31.766+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:32.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:31 smithi123 conmon[160185]: 2023-10-07T23:18:31.766+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:32.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:31 smithi123 conmon[160185]: 2023-10-07T23:18:31.766+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:32.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:32 smithi086 conmon[148156]: 2023-10-07T23:18:32.078+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:32.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:32 smithi086 conmon[148156]: 2023-10-07T23:18:32.078+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:32.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:31 smithi086 conmon[158442]: 2023-10-07T23:18:31.929+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:32.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:31 smithi086 conmon[158442]: 2023-10-07T23:18:31.929+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:32.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:31 smithi086 conmon[158442]: 2023-10-07T23:18:31.929+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:32.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:32 smithi067 conmon[120945]: 2023-10-07T23:18:32.187+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:32.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:32 smithi067 conmon[120945]: 2023-10-07T23:18:32.187+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:32.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:32 smithi123 conmon[150509]: 2023-10-07T23:18:32.302+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:32.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:32 smithi123 conmon[150509]: 2023-10-07T23:18:32.302+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:33.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:32 smithi067 conmon[130383]: 2023-10-07T23:18:32.693+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:33.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:32 smithi067 conmon[130383]: 2023-10-07T23:18:32.693+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:33.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:32 smithi067 conmon[130383]: 2023-10-07T23:18:32.693+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:33.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:32 smithi123 conmon[160185]: 2023-10-07T23:18:32.729+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:33.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:32 smithi123 conmon[160185]: 2023-10-07T23:18:32.729+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:33.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:32 smithi123 conmon[160185]: 2023-10-07T23:18:32.729+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:33.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-07T23:18:33.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:33.196 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-07T23:18:33.196 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-07T23:18:33.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.6 2023-10-07T23:18:33.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:33 smithi086 conmon[148156]: 2023-10-07T23:18:33.096+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:33.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:33 smithi086 conmon[148156]: 2023-10-07T23:18:33.096+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:33.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:32 smithi086 conmon[158442]: 2023-10-07T23:18:32.884+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:33.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:32 smithi086 conmon[158442]: 2023-10-07T23:18:32.884+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:33.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:32 smithi086 conmon[158442]: 2023-10-07T23:18:32.884+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:33.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[120945]: 2023-10-07T23:18:33.191+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:33.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[120945]: 2023-10-07T23:18:33.191+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:33.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:33 smithi067 systemd[1]: Stopping Ceph osd.6 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:33.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:33 smithi123 conmon[150509]: 2023-10-07T23:18:33.257+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:33.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:33 smithi123 conmon[150509]: 2023-10-07T23:18:33.257+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:34.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[130383]: 2023-10-07T23:18:33.738+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:34.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[130383]: 2023-10-07T23:18:33.739+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:34.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[130383]: 2023-10-07T23:18:33.739+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:34.005 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[120945]: 2023-10-07T23:18:33.720+0000 7fe17d179700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:34.005 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[120945]: 2023-10-07T23:18:33.720+0000 7fe17d179700 -1 osd.6 97 *** Got signal Terminated *** 2023-10-07T23:18:34.005 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:33 smithi067 conmon[120945]: 2023-10-07T23:18:33.720+0000 7fe17d179700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:34.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:33 smithi123 conmon[160185]: 2023-10-07T23:18:33.720+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:34.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:33 smithi123 conmon[160185]: 2023-10-07T23:18:33.720+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:34.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:33 smithi123 conmon[160185]: 2023-10-07T23:18:33.720+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:34.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:34 smithi086 conmon[148156]: 2023-10-07T23:18:34.094+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:34.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:34 smithi086 conmon[148156]: 2023-10-07T23:18:34.094+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:34.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:33 smithi086 conmon[158442]: 2023-10-07T23:18:33.881+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:34.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:33 smithi086 conmon[158442]: 2023-10-07T23:18:33.881+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:34.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:33 smithi086 conmon[158442]: 2023-10-07T23:18:33.881+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:34.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:34 smithi067 conmon[120945]: 2023-10-07T23:18:34.179+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:34.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:34 smithi067 conmon[120945]: 2023-10-07T23:18:34.179+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:34.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:34 smithi123 conmon[150509]: 2023-10-07T23:18:34.220+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:34.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:34 smithi123 conmon[150509]: 2023-10-07T23:18:34.220+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:35.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:34 smithi123 conmon[160185]: 2023-10-07T23:18:34.698+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:35.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:34 smithi123 conmon[160185]: 2023-10-07T23:18:34.698+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:35.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:34 smithi123 conmon[160185]: 2023-10-07T23:18:34.698+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:35.186 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:34 smithi067 conmon[130383]: 2023-10-07T23:18:34.782+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:35.186 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:34 smithi067 conmon[130383]: 2023-10-07T23:18:34.782+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:35.187 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:34 smithi067 conmon[130383]: 2023-10-07T23:18:34.782+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:35.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:35 smithi086 conmon[148156]: 2023-10-07T23:18:35.069+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:35.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:35 smithi086 conmon[148156]: 2023-10-07T23:18:35.069+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:35.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:34 smithi086 conmon[158442]: 2023-10-07T23:18:34.930+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:35.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:34 smithi086 conmon[158442]: 2023-10-07T23:18:34.930+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:35.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:34 smithi086 conmon[158442]: 2023-10-07T23:18:34.930+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:35.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:35 smithi067 conmon[120945]: 2023-10-07T23:18:35.185+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:35.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:35 smithi067 conmon[120945]: 2023-10-07T23:18:35.185+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:35.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:35 smithi123 conmon[150509]: 2023-10-07T23:18:35.179+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:35.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:35 smithi123 conmon[150509]: 2023-10-07T23:18:35.179+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:36.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:35 smithi067 conmon[130383]: 2023-10-07T23:18:35.746+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:36.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:35 smithi067 conmon[130383]: 2023-10-07T23:18:35.746+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:36.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:35 smithi067 conmon[130383]: 2023-10-07T23:18:35.746+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:36.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:35 smithi123 conmon[160185]: 2023-10-07T23:18:35.670+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:36.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:35 smithi123 conmon[160185]: 2023-10-07T23:18:35.670+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:36.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:35 smithi123 conmon[160185]: 2023-10-07T23:18:35.670+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:36.198 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:36 smithi086 conmon[148156]: 2023-10-07T23:18:36.078+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:36.198 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:36 smithi086 conmon[148156]: 2023-10-07T23:18:36.078+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:35 smithi086 conmon[158442]: 2023-10-07T23:18:35.910+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:36.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:35 smithi086 conmon[158442]: 2023-10-07T23:18:35.910+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:36.200 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:35 smithi086 conmon[158442]: 2023-10-07T23:18:35.910+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:36.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:36 smithi067 conmon[120945]: 2023-10-07T23:18:36.178+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:36.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:36 smithi067 conmon[120945]: 2023-10-07T23:18:36.178+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:36.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:36 smithi067 conmon[120945]: 2023-10-07T23:18:36.178+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:10.744533+0000 front 2023-10-07T23:18:10.744592+0000 (oldest deadline 2023-10-07T23:18:36.044567+0000) 2023-10-07T23:18:36.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:36 smithi123 conmon[150509]: 2023-10-07T23:18:36.162+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:36.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:36 smithi123 conmon[150509]: 2023-10-07T23:18:36.162+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:37.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:36 smithi067 conmon[130383]: 2023-10-07T23:18:36.708+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:37.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:36 smithi067 conmon[130383]: 2023-10-07T23:18:36.708+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:37.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:36 smithi067 conmon[130383]: 2023-10-07T23:18:36.708+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:37.121 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:36 smithi123 conmon[160185]: 2023-10-07T23:18:36.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:37.121 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:36 smithi123 conmon[160185]: 2023-10-07T23:18:36.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:37.122 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:36 smithi123 conmon[160185]: 2023-10-07T23:18:36.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:37.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:37 smithi123 conmon[150509]: 2023-10-07T23:18:37.119+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:37.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:37 smithi123 conmon[150509]: 2023-10-07T23:18:37.119+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:37.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[148156]: 2023-10-07T23:18:37.071+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:37.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[148156]: 2023-10-07T23:18:37.071+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:37.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[148156]: 2023-10-07T23:18:37.071+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:37.448 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:36 smithi086 conmon[158442]: 2023-10-07T23:18:36.949+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:37.448 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:36 smithi086 conmon[158442]: 2023-10-07T23:18:36.949+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:37.448 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:36 smithi086 conmon[158442]: 2023-10-07T23:18:36.949+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:37.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:37 smithi067 conmon[120945]: 2023-10-07T23:18:37.214+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:37.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:37 smithi067 conmon[120945]: 2023-10-07T23:18:37.214+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:37.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:37 smithi067 conmon[120945]: 2023-10-07T23:18:37.214+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:10.744533+0000 front 2023-10-07T23:18:10.744592+0000 (oldest deadline 2023-10-07T23:18:36.044567+0000) 2023-10-07T23:18:38.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:37 smithi067 conmon[130383]: 2023-10-07T23:18:37.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:38.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:37 smithi067 conmon[130383]: 2023-10-07T23:18:37.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:38.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:37 smithi067 conmon[130383]: 2023-10-07T23:18:37.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:38.111 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:37 smithi123 conmon[160185]: 2023-10-07T23:18:37.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:38.112 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:37 smithi123 conmon[160185]: 2023-10-07T23:18:37.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:38.112 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:37 smithi123 conmon[160185]: 2023-10-07T23:18:37.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:38.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:38 smithi123 conmon[150509]: 2023-10-07T23:18:38.111+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:38.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:38 smithi123 conmon[150509]: 2023-10-07T23:18:38.111+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:38.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[148156]: 2023-10-07T23:18:38.119+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:38.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[148156]: 2023-10-07T23:18:38.119+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:38.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[148156]: 2023-10-07T23:18:38.119+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:38.448 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[158442]: 2023-10-07T23:18:37.959+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:38.448 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[158442]: 2023-10-07T23:18:37.959+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:38.448 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[158442]: 2023-10-07T23:18:37.959+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:38.449 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:37 smithi086 conmon[158442]: 2023-10-07T23:18:37.959+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:38.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:38 smithi067 conmon[120945]: 2023-10-07T23:18:38.230+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:54.941492+0000 front 2023-10-07T23:17:54.941528+0000 (oldest deadline 2023-10-07T23:18:19.041715+0000) 2023-10-07T23:18:38.504 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:38 smithi067 conmon[120945]: 2023-10-07T23:18:38.230+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.143317+0000 front 2023-10-07T23:18:03.143468+0000 (oldest deadline 2023-10-07T23:18:29.043473+0000) 2023-10-07T23:18:38.505 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:38 smithi067 conmon[120945]: 2023-10-07T23:18:38.230+0000 7fe17a779700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:10.744533+0000 front 2023-10-07T23:18:10.744592+0000 (oldest deadline 2023-10-07T23:18:36.044567+0000) 2023-10-07T23:18:38.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:38 smithi123 conmon[160185]: 2023-10-07T23:18:38.610+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:38.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:38 smithi123 conmon[160185]: 2023-10-07T23:18:38.610+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:38.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:38 smithi123 conmon[160185]: 2023-10-07T23:18:38.610+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:39.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[148156]: 2023-10-07T23:18:39.104+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:39.198 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[148156]: 2023-10-07T23:18:39.104+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:39.198 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[148156]: 2023-10-07T23:18:39.104+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:39.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[158442]: 2023-10-07T23:18:38.911+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:39.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[158442]: 2023-10-07T23:18:38.911+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:39.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[158442]: 2023-10-07T23:18:38.911+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:39.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:38 smithi086 conmon[158442]: 2023-10-07T23:18:38.911+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:39.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:38 smithi067 conmon[130383]: 2023-10-07T23:18:38.772+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:39.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:38 smithi067 conmon[130383]: 2023-10-07T23:18:38.772+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:39.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:38 smithi067 conmon[130383]: 2023-10-07T23:18:38.772+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:39.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:39 smithi123 conmon[150509]: 2023-10-07T23:18:39.143+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:39.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:39 smithi123 conmon[150509]: 2023-10-07T23:18:39.143+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:39.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:39 smithi123 conmon[150509]: 2023-10-07T23:18:39.143+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:39.736 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:39 smithi067 bash[165338]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-6 2023-10-07T23:18:40.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:39 smithi067 conmon[130383]: 2023-10-07T23:18:39.735+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:40.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:39 smithi067 conmon[130383]: 2023-10-07T23:18:39.735+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:40.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:39 smithi067 conmon[130383]: 2023-10-07T23:18:39.735+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:40.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:39 smithi067 conmon[130383]: 2023-10-07T23:18:39.735+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:40.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:39 smithi123 conmon[160185]: 2023-10-07T23:18:39.655+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:40.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:39 smithi123 conmon[160185]: 2023-10-07T23:18:39.655+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:40.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:39 smithi123 conmon[160185]: 2023-10-07T23:18:39.655+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:40.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[148156]: 2023-10-07T23:18:40.092+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:40.197 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[148156]: 2023-10-07T23:18:40.092+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:40.198 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[148156]: 2023-10-07T23:18:40.092+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:40.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[158442]: 2023-10-07T23:18:39.862+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:40.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[158442]: 2023-10-07T23:18:39.862+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:40.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[158442]: 2023-10-07T23:18:39.862+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:40.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:39 smithi086 conmon[158442]: 2023-10-07T23:18:39.862+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:40.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:40 smithi123 conmon[150509]: 2023-10-07T23:18:40.190+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:40.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:40 smithi123 conmon[150509]: 2023-10-07T23:18:40.190+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:40.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:40 smithi123 conmon[150509]: 2023-10-07T23:18:40.190+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:41.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:40 smithi067 conmon[130383]: 2023-10-07T23:18:40.706+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:41.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:40 smithi067 conmon[130383]: 2023-10-07T23:18:40.706+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:41.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:40 smithi067 conmon[130383]: 2023-10-07T23:18:40.706+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:41.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:40 smithi067 conmon[130383]: 2023-10-07T23:18:40.706+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:41.112 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[158442]: 2023-10-07T23:18:40.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:41.112 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[158442]: 2023-10-07T23:18:40.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:41.112 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[158442]: 2023-10-07T23:18:40.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:41.112 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:40 smithi086 conmon[158442]: 2023-10-07T23:18:40.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:41.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:40 smithi123 conmon[160185]: 2023-10-07T23:18:40.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:41.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:40 smithi123 conmon[160185]: 2023-10-07T23:18:40.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:41.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:40 smithi123 conmon[160185]: 2023-10-07T23:18:40.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:41.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-07T23:18:41.368 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:41 smithi067 systemd[1]: ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.6.service: Succeeded. 2023-10-07T23:18:41.368 INFO:journalctl@ceph.osd.6.smithi067.stdout:Oct 07 23:18:41 smithi067 systemd[1]: Stopped Ceph osd.6 for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:18:41.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:41.395 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-07T23:18:41.395 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-07T23:18:41.396 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.7 2023-10-07T23:18:41.423 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[148156]: 2023-10-07T23:18:41.111+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:41.423 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[148156]: 2023-10-07T23:18:41.111+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:41.423 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[148156]: 2023-10-07T23:18:41.111+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:41.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:41 smithi123 conmon[150509]: 2023-10-07T23:18:41.148+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:41.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:41 smithi123 conmon[150509]: 2023-10-07T23:18:41.148+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:41.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:41 smithi123 conmon[150509]: 2023-10-07T23:18:41.148+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:41.697 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 systemd[1]: Stopping Ceph osd.7 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:42.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:41 smithi067 conmon[130383]: 2023-10-07T23:18:41.709+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:42.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:41 smithi067 conmon[130383]: 2023-10-07T23:18:41.709+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:42.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:41 smithi067 conmon[130383]: 2023-10-07T23:18:41.709+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:42.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:41 smithi067 conmon[130383]: 2023-10-07T23:18:41.709+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:42.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:41 smithi123 conmon[160185]: 2023-10-07T23:18:41.699+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:42.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:41 smithi123 conmon[160185]: 2023-10-07T23:18:41.699+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:42.141 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:41 smithi123 conmon[160185]: 2023-10-07T23:18:41.699+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:42.141 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[148156]: 2023-10-07T23:18:41.751+0000 7efde0664700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:42.142 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[148156]: 2023-10-07T23:18:41.751+0000 7efde0664700 -1 osd.7 97 *** Got signal Terminated *** 2023-10-07T23:18:42.142 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[148156]: 2023-10-07T23:18:41.751+0000 7efde0664700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:42.142 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[158442]: 2023-10-07T23:18:41.774+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:42.142 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[158442]: 2023-10-07T23:18:41.774+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:42.142 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[158442]: 2023-10-07T23:18:41.774+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:42.142 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:41 smithi086 conmon[158442]: 2023-10-07T23:18:41.774+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:42.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[148156]: 2023-10-07T23:18:42.139+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:42.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[148156]: 2023-10-07T23:18:42.139+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:42.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[148156]: 2023-10-07T23:18:42.139+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:42.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:42 smithi123 conmon[150509]: 2023-10-07T23:18:42.140+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:42.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:42 smithi123 conmon[150509]: 2023-10-07T23:18:42.140+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:42.641 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:42 smithi123 conmon[150509]: 2023-10-07T23:18:42.140+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:43.108 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:42 smithi123 conmon[160185]: 2023-10-07T23:18:42.656+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:43.109 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:42 smithi123 conmon[160185]: 2023-10-07T23:18:42.656+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:43.109 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:42 smithi123 conmon[160185]: 2023-10-07T23:18:42.656+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:43.128 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[158442]: 2023-10-07T23:18:42.796+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:43.128 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[158442]: 2023-10-07T23:18:42.796+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:43.128 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[158442]: 2023-10-07T23:18:42.796+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:43.128 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:42 smithi086 conmon[158442]: 2023-10-07T23:18:42.796+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:43.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:42 smithi067 conmon[130383]: 2023-10-07T23:18:42.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:43.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:42 smithi067 conmon[130383]: 2023-10-07T23:18:42.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:43.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:42 smithi067 conmon[130383]: 2023-10-07T23:18:42.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:43.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:42 smithi067 conmon[130383]: 2023-10-07T23:18:42.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:43.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:43 smithi123 conmon[150509]: 2023-10-07T23:18:43.108+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:43.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:43 smithi123 conmon[150509]: 2023-10-07T23:18:43.108+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:43.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:43 smithi123 conmon[150509]: 2023-10-07T23:18:43.108+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:43.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[148156]: 2023-10-07T23:18:43.127+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:43.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[148156]: 2023-10-07T23:18:43.127+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:43.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[148156]: 2023-10-07T23:18:43.127+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:44.061 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:43 smithi123 conmon[160185]: 2023-10-07T23:18:43.674+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:44.062 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:43 smithi123 conmon[160185]: 2023-10-07T23:18:43.674+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:44.062 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:43 smithi123 conmon[160185]: 2023-10-07T23:18:43.674+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:44.133 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[158442]: 2023-10-07T23:18:43.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:44.133 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[158442]: 2023-10-07T23:18:43.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:44.134 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[158442]: 2023-10-07T23:18:43.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:44.134 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:43 smithi086 conmon[158442]: 2023-10-07T23:18:43.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:44.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:43 smithi067 conmon[130383]: 2023-10-07T23:18:43.786+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:44.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:43 smithi067 conmon[130383]: 2023-10-07T23:18:43.786+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:44.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:43 smithi067 conmon[130383]: 2023-10-07T23:18:43.786+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:44.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:43 smithi067 conmon[130383]: 2023-10-07T23:18:43.786+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:44.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[150509]: 2023-10-07T23:18:44.061+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:44.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[150509]: 2023-10-07T23:18:44.061+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:44.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[150509]: 2023-10-07T23:18:44.061+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:44.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[148156]: 2023-10-07T23:18:44.132+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:44.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[148156]: 2023-10-07T23:18:44.132+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:44.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[148156]: 2023-10-07T23:18:44.132+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:44.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[148156]: 2023-10-07T23:18:44.132+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.441430+0000 front 2023-10-07T23:18:20.441400+0000 (oldest deadline 2023-10-07T23:18:43.341551+0000) 2023-10-07T23:18:45.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:44 smithi067 conmon[130383]: 2023-10-07T23:18:44.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:45.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:44 smithi067 conmon[130383]: 2023-10-07T23:18:44.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:45.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:44 smithi067 conmon[130383]: 2023-10-07T23:18:44.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:45.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:44 smithi067 conmon[130383]: 2023-10-07T23:18:44.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:45.005 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:44 smithi067 conmon[130383]: 2023-10-07T23:18:44.752+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:45.068 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[160185]: 2023-10-07T23:18:44.675+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:45.068 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[160185]: 2023-10-07T23:18:44.675+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:45.068 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[160185]: 2023-10-07T23:18:44.675+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:45.068 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:44 smithi123 conmon[160185]: 2023-10-07T23:18:44.675+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:45.158 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[158442]: 2023-10-07T23:18:44.813+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:45.159 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[158442]: 2023-10-07T23:18:44.813+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:45.159 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[158442]: 2023-10-07T23:18:44.813+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:45.159 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:44 smithi086 conmon[158442]: 2023-10-07T23:18:44.813+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:45.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[150509]: 2023-10-07T23:18:45.067+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:45.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[150509]: 2023-10-07T23:18:45.067+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:45.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[150509]: 2023-10-07T23:18:45.067+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:45.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[150509]: 2023-10-07T23:18:45.067+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:45.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[148156]: 2023-10-07T23:18:45.157+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:45.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[148156]: 2023-10-07T23:18:45.157+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:45.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[148156]: 2023-10-07T23:18:45.157+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:45.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[148156]: 2023-10-07T23:18:45.157+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.441430+0000 front 2023-10-07T23:18:20.441400+0000 (oldest deadline 2023-10-07T23:18:43.341551+0000) 2023-10-07T23:18:46.048 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[160185]: 2023-10-07T23:18:45.672+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:46.049 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[160185]: 2023-10-07T23:18:45.672+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:46.049 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[160185]: 2023-10-07T23:18:45.672+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:46.049 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:45 smithi123 conmon[160185]: 2023-10-07T23:18:45.672+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:46.185 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[158442]: 2023-10-07T23:18:45.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:46.185 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[158442]: 2023-10-07T23:18:45.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:46.185 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[158442]: 2023-10-07T23:18:45.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:46.185 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[158442]: 2023-10-07T23:18:45.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:46.186 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:45 smithi086 conmon[158442]: 2023-10-07T23:18:45.854+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:46.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:45 smithi067 conmon[130383]: 2023-10-07T23:18:45.774+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:46.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:45 smithi067 conmon[130383]: 2023-10-07T23:18:45.774+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:46.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:45 smithi067 conmon[130383]: 2023-10-07T23:18:45.774+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:46.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:45 smithi067 conmon[130383]: 2023-10-07T23:18:45.774+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:46.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:45 smithi067 conmon[130383]: 2023-10-07T23:18:45.774+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:46.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[150509]: 2023-10-07T23:18:46.048+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:46.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[150509]: 2023-10-07T23:18:46.048+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:46.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[150509]: 2023-10-07T23:18:46.048+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:46.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[150509]: 2023-10-07T23:18:46.048+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:46.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[148156]: 2023-10-07T23:18:46.183+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.737360+0000 front 2023-10-07T23:17:57.737381+0000 (oldest deadline 2023-10-07T23:18:22.437578+0000) 2023-10-07T23:18:46.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[148156]: 2023-10-07T23:18:46.184+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.738987+0000 front 2023-10-07T23:18:04.738787+0000 (oldest deadline 2023-10-07T23:18:28.838966+0000) 2023-10-07T23:18:46.447 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[148156]: 2023-10-07T23:18:46.184+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.340060+0000 front 2023-10-07T23:18:12.340123+0000 (oldest deadline 2023-10-07T23:18:37.040091+0000) 2023-10-07T23:18:46.448 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[148156]: 2023-10-07T23:18:46.184+0000 7efdde465700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.441430+0000 front 2023-10-07T23:18:20.441400+0000 (oldest deadline 2023-10-07T23:18:43.341551+0000) 2023-10-07T23:18:47.040 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[160185]: 2023-10-07T23:18:46.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:47.041 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[160185]: 2023-10-07T23:18:46.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:47.041 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[160185]: 2023-10-07T23:18:46.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:47.041 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:46 smithi123 conmon[160185]: 2023-10-07T23:18:46.676+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:47.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[158442]: 2023-10-07T23:18:46.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:47.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[158442]: 2023-10-07T23:18:46.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:47.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[158442]: 2023-10-07T23:18:46.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:47.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[158442]: 2023-10-07T23:18:46.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:47.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:46 smithi086 conmon[158442]: 2023-10-07T23:18:46.820+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:47.256 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:46 smithi067 conmon[130383]: 2023-10-07T23:18:46.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:47.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:46 smithi067 conmon[130383]: 2023-10-07T23:18:46.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:47.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:46 smithi067 conmon[130383]: 2023-10-07T23:18:46.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:47.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:46 smithi067 conmon[130383]: 2023-10-07T23:18:46.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:47.261 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:46 smithi067 conmon[130383]: 2023-10-07T23:18:46.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:47.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[150509]: 2023-10-07T23:18:47.040+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:47.392 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[150509]: 2023-10-07T23:18:47.040+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:47.392 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[150509]: 2023-10-07T23:18:47.040+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:47.392 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[150509]: 2023-10-07T23:18:47.040+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:47.947 INFO:journalctl@ceph.osd.7.smithi086.stdout:Oct 07 23:18:47 smithi086 bash[171329]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-7 2023-10-07T23:18:47.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:47 smithi086 conmon[158442]: 2023-10-07T23:18:47.818+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:47.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:47 smithi086 conmon[158442]: 2023-10-07T23:18:47.818+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:47.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:47 smithi086 conmon[158442]: 2023-10-07T23:18:47.818+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:47.949 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:47 smithi086 conmon[158442]: 2023-10-07T23:18:47.818+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:47.949 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:47 smithi086 conmon[158442]: 2023-10-07T23:18:47.818+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:48.065 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[160185]: 2023-10-07T23:18:47.694+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:48.065 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[160185]: 2023-10-07T23:18:47.694+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:48.065 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[160185]: 2023-10-07T23:18:47.694+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:48.065 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:47 smithi123 conmon[160185]: 2023-10-07T23:18:47.694+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:48.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:47 smithi067 conmon[130383]: 2023-10-07T23:18:47.792+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:48.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:47 smithi067 conmon[130383]: 2023-10-07T23:18:47.792+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:48.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:47 smithi067 conmon[130383]: 2023-10-07T23:18:47.792+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:48.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:47 smithi067 conmon[130383]: 2023-10-07T23:18:47.792+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:48.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:47 smithi067 conmon[130383]: 2023-10-07T23:18:47.792+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:48.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[150509]: 2023-10-07T23:18:48.064+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:48.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[150509]: 2023-10-07T23:18:48.064+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:48.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[150509]: 2023-10-07T23:18:48.064+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:48.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[150509]: 2023-10-07T23:18:48.064+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:48.947 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:48 smithi086 conmon[158442]: 2023-10-07T23:18:48.791+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:48.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:48 smithi086 conmon[158442]: 2023-10-07T23:18:48.791+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:48.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:48 smithi086 conmon[158442]: 2023-10-07T23:18:48.791+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:48.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:48 smithi086 conmon[158442]: 2023-10-07T23:18:48.791+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:48.948 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:48 smithi086 conmon[158442]: 2023-10-07T23:18:48.791+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:49.088 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[160185]: 2023-10-07T23:18:48.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:49.089 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[160185]: 2023-10-07T23:18:48.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:49.089 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[160185]: 2023-10-07T23:18:48.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:49.089 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:48 smithi123 conmon[160185]: 2023-10-07T23:18:48.665+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:49.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:48 smithi067 conmon[130383]: 2023-10-07T23:18:48.798+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:49.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:48 smithi067 conmon[130383]: 2023-10-07T23:18:48.798+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:49.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:48 smithi067 conmon[130383]: 2023-10-07T23:18:48.798+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:49.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:48 smithi067 conmon[130383]: 2023-10-07T23:18:48.798+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:49.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:48 smithi067 conmon[130383]: 2023-10-07T23:18:48.798+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:49.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-07T23:18:49.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.089+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:49.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.089+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:49.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.089+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:49.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.089+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:49.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:49.432 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-07T23:18:49.432 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-07T23:18:49.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.8 2023-10-07T23:18:49.785 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 systemd[1]: Stopping Ceph osd.8 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:49.785 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.784+0000 7f05b6899700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:49.785 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[160185]: 2023-10-07T23:18:49.630+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:49.785 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[160185]: 2023-10-07T23:18:49.630+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:49.785 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[160185]: 2023-10-07T23:18:49.630+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:49.786 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[160185]: 2023-10-07T23:18:49.630+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:50.118 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.784+0000 7f05b6899700 -1 osd.8 97 *** Got signal Terminated *** 2023-10-07T23:18:50.119 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:49 smithi123 conmon[150509]: 2023-10-07T23:18:49.784+0000 7f05b6899700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:50.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:49 smithi086 conmon[158442]: 2023-10-07T23:18:49.782+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:50.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:49 smithi086 conmon[158442]: 2023-10-07T23:18:49.782+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:50.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:49 smithi086 conmon[158442]: 2023-10-07T23:18:49.782+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:50.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:49 smithi086 conmon[158442]: 2023-10-07T23:18:49.782+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:50.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:49 smithi086 conmon[158442]: 2023-10-07T23:18:49.782+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:50.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:49 smithi067 conmon[130383]: 2023-10-07T23:18:49.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:50.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:49 smithi067 conmon[130383]: 2023-10-07T23:18:49.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:50.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:49 smithi067 conmon[130383]: 2023-10-07T23:18:49.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:50.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:49 smithi067 conmon[130383]: 2023-10-07T23:18:49.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:50.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:49 smithi067 conmon[130383]: 2023-10-07T23:18:49.755+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:50.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[150509]: 2023-10-07T23:18:50.119+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:50.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[150509]: 2023-10-07T23:18:50.119+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:50.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[150509]: 2023-10-07T23:18:50.119+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:50.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[150509]: 2023-10-07T23:18:50.119+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:50.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[160185]: 2023-10-07T23:18:50.585+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:50.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[160185]: 2023-10-07T23:18:50.585+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:50.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[160185]: 2023-10-07T23:18:50.585+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:50.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:50 smithi123 conmon[160185]: 2023-10-07T23:18:50.585+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:51.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:50 smithi086 conmon[158442]: 2023-10-07T23:18:50.799+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:51.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:50 smithi086 conmon[158442]: 2023-10-07T23:18:50.799+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:51.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:50 smithi086 conmon[158442]: 2023-10-07T23:18:50.799+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:51.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:50 smithi086 conmon[158442]: 2023-10-07T23:18:50.799+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:51.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:50 smithi086 conmon[158442]: 2023-10-07T23:18:50.799+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:51.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:50 smithi067 conmon[130383]: 2023-10-07T23:18:50.805+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:51.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:50 smithi067 conmon[130383]: 2023-10-07T23:18:50.805+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:51.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:50 smithi067 conmon[130383]: 2023-10-07T23:18:50.805+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:51.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:50 smithi067 conmon[130383]: 2023-10-07T23:18:50.805+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:51.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:50 smithi067 conmon[130383]: 2023-10-07T23:18:50.805+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:51.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[150509]: 2023-10-07T23:18:51.082+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:51.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[150509]: 2023-10-07T23:18:51.082+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:51.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[150509]: 2023-10-07T23:18:51.082+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:51.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[150509]: 2023-10-07T23:18:51.082+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:51.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[160185]: 2023-10-07T23:18:51.615+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:51.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[160185]: 2023-10-07T23:18:51.615+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:51.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[160185]: 2023-10-07T23:18:51.615+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:51.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:51 smithi123 conmon[160185]: 2023-10-07T23:18:51.615+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:52.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:51 smithi086 conmon[158442]: 2023-10-07T23:18:51.806+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:52.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:51 smithi086 conmon[158442]: 2023-10-07T23:18:51.806+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:52.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:51 smithi086 conmon[158442]: 2023-10-07T23:18:51.806+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:52.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:51 smithi086 conmon[158442]: 2023-10-07T23:18:51.806+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:52.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:51 smithi086 conmon[158442]: 2023-10-07T23:18:51.806+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:52.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:51 smithi086 conmon[158442]: 2023-10-07T23:18:51.806+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:52.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:51 smithi067 conmon[130383]: 2023-10-07T23:18:51.793+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:52.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:51 smithi067 conmon[130383]: 2023-10-07T23:18:51.793+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:52.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:51 smithi067 conmon[130383]: 2023-10-07T23:18:51.793+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:52.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:51 smithi067 conmon[130383]: 2023-10-07T23:18:51.793+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:52.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:51 smithi067 conmon[130383]: 2023-10-07T23:18:51.793+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:52.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[150509]: 2023-10-07T23:18:52.041+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:52.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[150509]: 2023-10-07T23:18:52.041+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:52.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[150509]: 2023-10-07T23:18:52.041+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:52.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[150509]: 2023-10-07T23:18:52.041+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:52.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[160185]: 2023-10-07T23:18:52.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:52.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[160185]: 2023-10-07T23:18:52.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:52.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[160185]: 2023-10-07T23:18:52.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:52.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:52 smithi123 conmon[160185]: 2023-10-07T23:18:52.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:53.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:52 smithi086 conmon[158442]: 2023-10-07T23:18:52.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:53.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:52 smithi086 conmon[158442]: 2023-10-07T23:18:52.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:53.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:52 smithi086 conmon[158442]: 2023-10-07T23:18:52.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:53.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:52 smithi086 conmon[158442]: 2023-10-07T23:18:52.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:53.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:52 smithi086 conmon[158442]: 2023-10-07T23:18:52.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:53.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:52 smithi086 conmon[158442]: 2023-10-07T23:18:52.838+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:53.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:52 smithi067 conmon[130383]: 2023-10-07T23:18:52.813+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:53.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:52 smithi067 conmon[130383]: 2023-10-07T23:18:52.813+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:53.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:52 smithi067 conmon[130383]: 2023-10-07T23:18:52.813+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:53.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:52 smithi067 conmon[130383]: 2023-10-07T23:18:52.813+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:53.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:52 smithi067 conmon[130383]: 2023-10-07T23:18:52.813+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:53.390 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[150509]: 2023-10-07T23:18:53.019+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:53.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[150509]: 2023-10-07T23:18:53.019+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:53.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[150509]: 2023-10-07T23:18:53.019+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:53.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[150509]: 2023-10-07T23:18:53.019+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:53.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[160185]: 2023-10-07T23:18:53.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:53.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[160185]: 2023-10-07T23:18:53.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:53.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[160185]: 2023-10-07T23:18:53.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:53.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:53 smithi123 conmon[160185]: 2023-10-07T23:18:53.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:54.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:53 smithi086 conmon[158442]: 2023-10-07T23:18:53.808+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:54.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:53 smithi086 conmon[158442]: 2023-10-07T23:18:53.808+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:54.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:53 smithi086 conmon[158442]: 2023-10-07T23:18:53.808+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:54.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:53 smithi086 conmon[158442]: 2023-10-07T23:18:53.808+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:54.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:53 smithi086 conmon[158442]: 2023-10-07T23:18:53.808+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:54.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:53 smithi086 conmon[158442]: 2023-10-07T23:18:53.808+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:54.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:53 smithi067 conmon[130383]: 2023-10-07T23:18:53.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:54.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:53 smithi067 conmon[130383]: 2023-10-07T23:18:53.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:54.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:53 smithi067 conmon[130383]: 2023-10-07T23:18:53.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:54.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:53 smithi067 conmon[130383]: 2023-10-07T23:18:53.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:54.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:53 smithi067 conmon[130383]: 2023-10-07T23:18:53.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:54.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:53 smithi067 conmon[130383]: 2023-10-07T23:18:53.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:18:54.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[150509]: 2023-10-07T23:18:54.069+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:55.459423+0000 front 2023-10-07T23:17:55.459450+0000 (oldest deadline 2023-10-07T23:18:20.759438+0000) 2023-10-07T23:18:54.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[150509]: 2023-10-07T23:18:54.069+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:00.759817+0000 front 2023-10-07T23:18:00.759795+0000 (oldest deadline 2023-10-07T23:18:26.659938+0000) 2023-10-07T23:18:54.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[150509]: 2023-10-07T23:18:54.069+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:14.062325+0000 front 2023-10-07T23:18:14.062344+0000 (oldest deadline 2023-10-07T23:18:38.762524+0000) 2023-10-07T23:18:54.391 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[150509]: 2023-10-07T23:18:54.069+0000 7f05b0692700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:18.763138+0000 front 2023-10-07T23:18:18.763273+0000 (oldest deadline 2023-10-07T23:18:44.663245+0000) 2023-10-07T23:18:54.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[160185]: 2023-10-07T23:18:54.543+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:54.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[160185]: 2023-10-07T23:18:54.543+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:54.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[160185]: 2023-10-07T23:18:54.543+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:54.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:54 smithi123 conmon[160185]: 2023-10-07T23:18:54.543+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:55.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:54 smithi086 conmon[158442]: 2023-10-07T23:18:54.807+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:55.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:54 smithi086 conmon[158442]: 2023-10-07T23:18:54.807+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:55.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:54 smithi086 conmon[158442]: 2023-10-07T23:18:54.807+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:55.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:54 smithi086 conmon[158442]: 2023-10-07T23:18:54.807+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:55.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:54 smithi086 conmon[158442]: 2023-10-07T23:18:54.807+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:55.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:54 smithi086 conmon[158442]: 2023-10-07T23:18:54.807+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:55.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:54 smithi067 conmon[130383]: 2023-10-07T23:18:54.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:55.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:54 smithi067 conmon[130383]: 2023-10-07T23:18:54.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:55.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:54 smithi067 conmon[130383]: 2023-10-07T23:18:54.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:55.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:54 smithi067 conmon[130383]: 2023-10-07T23:18:54.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:55.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:54 smithi067 conmon[130383]: 2023-10-07T23:18:54.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:55.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:54 smithi067 conmon[130383]: 2023-10-07T23:18:54.799+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:18:55.640 INFO:journalctl@ceph.osd.8.smithi123.stdout:Oct 07 23:18:55 smithi123 bash[173234]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-8 2023-10-07T23:18:55.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:55 smithi123 conmon[160185]: 2023-10-07T23:18:55.511+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:55.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:55 smithi123 conmon[160185]: 2023-10-07T23:18:55.511+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:55.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:55 smithi123 conmon[160185]: 2023-10-07T23:18:55.511+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:55.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:55 smithi123 conmon[160185]: 2023-10-07T23:18:55.511+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:56.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:55 smithi086 conmon[158442]: 2023-10-07T23:18:55.795+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:56.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:55 smithi086 conmon[158442]: 2023-10-07T23:18:55.795+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:56.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:55 smithi086 conmon[158442]: 2023-10-07T23:18:55.795+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:56.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:55 smithi086 conmon[158442]: 2023-10-07T23:18:55.795+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:56.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:55 smithi086 conmon[158442]: 2023-10-07T23:18:55.795+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:56.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:55 smithi086 conmon[158442]: 2023-10-07T23:18:55.795+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:56.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:55 smithi067 conmon[130383]: 2023-10-07T23:18:55.802+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:56.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:55 smithi067 conmon[130383]: 2023-10-07T23:18:55.802+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:56.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:55 smithi067 conmon[130383]: 2023-10-07T23:18:55.802+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:56.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:55 smithi067 conmon[130383]: 2023-10-07T23:18:55.802+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:56.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:55 smithi067 conmon[130383]: 2023-10-07T23:18:55.802+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:56.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:55 smithi067 conmon[130383]: 2023-10-07T23:18:55.802+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:18:56.791 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:56 smithi123 conmon[160185]: 2023-10-07T23:18:56.506+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:56.792 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:56 smithi123 conmon[160185]: 2023-10-07T23:18:56.506+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:56.792 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:56 smithi123 conmon[160185]: 2023-10-07T23:18:56.506+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:56.792 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:56 smithi123 conmon[160185]: 2023-10-07T23:18:56.506+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:56.792 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:56 smithi123 conmon[160185]: 2023-10-07T23:18:56.506+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:18:57.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:56 smithi086 conmon[158442]: 2023-10-07T23:18:56.785+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:57.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:56 smithi086 conmon[158442]: 2023-10-07T23:18:56.785+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:57.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:56 smithi086 conmon[158442]: 2023-10-07T23:18:56.785+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:57.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:56 smithi086 conmon[158442]: 2023-10-07T23:18:56.785+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:57.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:56 smithi086 conmon[158442]: 2023-10-07T23:18:56.785+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:57.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:56 smithi086 conmon[158442]: 2023-10-07T23:18:56.785+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:57.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:56 smithi067 conmon[130383]: 2023-10-07T23:18:56.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:57.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:56 smithi067 conmon[130383]: 2023-10-07T23:18:56.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:57.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:56 smithi067 conmon[130383]: 2023-10-07T23:18:56.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:57.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:56 smithi067 conmon[130383]: 2023-10-07T23:18:56.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:57.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:56 smithi067 conmon[130383]: 2023-10-07T23:18:56.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:57.256 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:56 smithi067 conmon[130383]: 2023-10-07T23:18:56.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:18:57.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-07T23:18:57.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:18:57.384 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-07T23:18:57.384 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-07T23:18:57.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.9 2023-10-07T23:18:57.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:57 smithi123 conmon[160185]: 2023-10-07T23:18:57.534+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:57.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:57 smithi123 conmon[160185]: 2023-10-07T23:18:57.534+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:57.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:57 smithi123 conmon[160185]: 2023-10-07T23:18:57.534+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:57.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:57 smithi123 conmon[160185]: 2023-10-07T23:18:57.534+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:57.641 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:57 smithi123 conmon[160185]: 2023-10-07T23:18:57.534+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:18:57.755 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 systemd[1]: Stopping Ceph osd.9 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:18:58.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:57 smithi086 conmon[158442]: 2023-10-07T23:18:57.804+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:58.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:57 smithi086 conmon[158442]: 2023-10-07T23:18:57.804+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:58.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:57 smithi086 conmon[158442]: 2023-10-07T23:18:57.804+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:58.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:57 smithi086 conmon[158442]: 2023-10-07T23:18:57.804+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:58.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:57 smithi086 conmon[158442]: 2023-10-07T23:18:57.804+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:58.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:57 smithi086 conmon[158442]: 2023-10-07T23:18:57.804+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:58.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.804+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:58.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.804+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:58.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.804+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:58.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.804+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:58.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.804+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:58.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.804+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:18:58.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.908+0000 7f2147a47700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:18:58.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.908+0000 7f2147a47700 -1 osd.9 97 *** Got signal Terminated *** 2023-10-07T23:18:58.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:57 smithi067 conmon[130383]: 2023-10-07T23:18:57.908+0000 7f2147a47700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:18:58.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:58 smithi123 conmon[160185]: 2023-10-07T23:18:58.502+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:58.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:58 smithi123 conmon[160185]: 2023-10-07T23:18:58.502+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:58.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:58 smithi123 conmon[160185]: 2023-10-07T23:18:58.502+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:58.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:58 smithi123 conmon[160185]: 2023-10-07T23:18:58.502+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:58.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:58 smithi123 conmon[160185]: 2023-10-07T23:18:58.502+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:18:59.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:58 smithi086 conmon[158442]: 2023-10-07T23:18:58.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:18:59.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:58 smithi086 conmon[158442]: 2023-10-07T23:18:58.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:18:59.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:58 smithi086 conmon[158442]: 2023-10-07T23:18:58.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:18:59.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:58 smithi086 conmon[158442]: 2023-10-07T23:18:58.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:18:59.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:58 smithi086 conmon[158442]: 2023-10-07T23:18:58.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:18:59.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:58 smithi086 conmon[158442]: 2023-10-07T23:18:58.788+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:18:59.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:58 smithi067 conmon[130383]: 2023-10-07T23:18:58.842+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:18:59.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:58 smithi067 conmon[130383]: 2023-10-07T23:18:58.842+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:18:59.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:58 smithi067 conmon[130383]: 2023-10-07T23:18:58.842+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:18:59.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:58 smithi067 conmon[130383]: 2023-10-07T23:18:58.842+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:18:59.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:58 smithi067 conmon[130383]: 2023-10-07T23:18:58.842+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:18:59.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:58 smithi067 conmon[130383]: 2023-10-07T23:18:58.842+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:18:59.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:59 smithi123 conmon[160185]: 2023-10-07T23:18:59.508+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:18:59.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:59 smithi123 conmon[160185]: 2023-10-07T23:18:59.508+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:18:59.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:59 smithi123 conmon[160185]: 2023-10-07T23:18:59.508+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:18:59.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:59 smithi123 conmon[160185]: 2023-10-07T23:18:59.508+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:18:59.892 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:18:59 smithi123 conmon[160185]: 2023-10-07T23:18:59.508+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:00.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:59 smithi086 conmon[158442]: 2023-10-07T23:18:59.776+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:00.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:59 smithi086 conmon[158442]: 2023-10-07T23:18:59.776+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:00.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:59 smithi086 conmon[158442]: 2023-10-07T23:18:59.776+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:00.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:59 smithi086 conmon[158442]: 2023-10-07T23:18:59.776+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:00.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:59 smithi086 conmon[158442]: 2023-10-07T23:18:59.776+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:00.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:18:59 smithi086 conmon[158442]: 2023-10-07T23:18:59.776+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:00.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:59 smithi067 conmon[130383]: 2023-10-07T23:18:59.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:19:00.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:59 smithi067 conmon[130383]: 2023-10-07T23:18:59.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:19:00.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:59 smithi067 conmon[130383]: 2023-10-07T23:18:59.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:19:00.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:59 smithi067 conmon[130383]: 2023-10-07T23:18:59.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:19:00.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:59 smithi067 conmon[130383]: 2023-10-07T23:18:59.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:19:00.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:18:59 smithi067 conmon[130383]: 2023-10-07T23:18:59.829+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:19:00.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:00 smithi123 conmon[160185]: 2023-10-07T23:19:00.516+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:00.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:00 smithi123 conmon[160185]: 2023-10-07T23:19:00.516+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:00.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:00 smithi123 conmon[160185]: 2023-10-07T23:19:00.516+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:00.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:00 smithi123 conmon[160185]: 2023-10-07T23:19:00.516+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:00.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:00 smithi123 conmon[160185]: 2023-10-07T23:19:00.516+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:00.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:00 smithi123 conmon[160185]: 2023-10-07T23:19:00.516+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:01.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:00 smithi086 conmon[158442]: 2023-10-07T23:19:00.766+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:01.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:00 smithi086 conmon[158442]: 2023-10-07T23:19:00.766+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:01.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:00 smithi086 conmon[158442]: 2023-10-07T23:19:00.766+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:01.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:00 smithi086 conmon[158442]: 2023-10-07T23:19:00.766+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:01.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:00 smithi086 conmon[158442]: 2023-10-07T23:19:00.766+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:01.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:00 smithi086 conmon[158442]: 2023-10-07T23:19:00.766+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:01.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:00 smithi067 conmon[130383]: 2023-10-07T23:19:00.816+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:19:01.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:00 smithi067 conmon[130383]: 2023-10-07T23:19:00.816+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:19:01.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:00 smithi067 conmon[130383]: 2023-10-07T23:19:00.816+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:19:01.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:00 smithi067 conmon[130383]: 2023-10-07T23:19:00.816+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:19:01.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:00 smithi067 conmon[130383]: 2023-10-07T23:19:00.816+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:19:01.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:00 smithi067 conmon[130383]: 2023-10-07T23:19:00.816+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:19:01.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:01 smithi123 conmon[160185]: 2023-10-07T23:19:01.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:01.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:01 smithi123 conmon[160185]: 2023-10-07T23:19:01.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:01.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:01 smithi123 conmon[160185]: 2023-10-07T23:19:01.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:01.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:01 smithi123 conmon[160185]: 2023-10-07T23:19:01.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:01.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:01 smithi123 conmon[160185]: 2023-10-07T23:19:01.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:01.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:01 smithi123 conmon[160185]: 2023-10-07T23:19:01.553+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:02.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:01 smithi086 conmon[158442]: 2023-10-07T23:19:01.801+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:02.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:01 smithi086 conmon[158442]: 2023-10-07T23:19:01.801+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:02.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:01 smithi086 conmon[158442]: 2023-10-07T23:19:01.801+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:02.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:01 smithi086 conmon[158442]: 2023-10-07T23:19:01.801+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:02.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:01 smithi086 conmon[158442]: 2023-10-07T23:19:01.801+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:02.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:01 smithi086 conmon[158442]: 2023-10-07T23:19:01.801+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:02.254 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:01 smithi067 conmon[130383]: 2023-10-07T23:19:01.859+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:19:02.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:01 smithi067 conmon[130383]: 2023-10-07T23:19:01.859+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:19:02.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:01 smithi067 conmon[130383]: 2023-10-07T23:19:01.859+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:19:02.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:01 smithi067 conmon[130383]: 2023-10-07T23:19:01.859+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:19:02.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:01 smithi067 conmon[130383]: 2023-10-07T23:19:01.859+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:19:02.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:01 smithi067 conmon[130383]: 2023-10-07T23:19:01.859+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:19:02.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:02 smithi123 conmon[160185]: 2023-10-07T23:19:02.552+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:02.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:02 smithi123 conmon[160185]: 2023-10-07T23:19:02.552+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:02.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:02 smithi123 conmon[160185]: 2023-10-07T23:19:02.552+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:02.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:02 smithi123 conmon[160185]: 2023-10-07T23:19:02.552+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:02.892 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:02 smithi123 conmon[160185]: 2023-10-07T23:19:02.552+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:02.892 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:02 smithi123 conmon[160185]: 2023-10-07T23:19:02.552+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:03.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:03.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:02 smithi086 conmon[158442]: 2023-10-07T23:19:02.844+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:03.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:02 smithi067 conmon[130383]: 2023-10-07T23:19:02.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:49.118483+0000 front 2023-10-07T23:17:49.118540+0000 (oldest deadline 2023-10-07T23:18:12.018762+0000) 2023-10-07T23:19:03.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:02 smithi067 conmon[130383]: 2023-10-07T23:19:02.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.621049+0000 front 2023-10-07T23:17:56.621153+0000 (oldest deadline 2023-10-07T23:18:22.520980+0000) 2023-10-07T23:19:03.255 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:02 smithi067 conmon[130383]: 2023-10-07T23:19:02.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:04.723111+0000 front 2023-10-07T23:18:04.723119+0000 (oldest deadline 2023-10-07T23:18:28.723927+0000) 2023-10-07T23:19:03.256 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:02 smithi067 conmon[130383]: 2023-10-07T23:19:02.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:13.425344+0000 front 2023-10-07T23:18:13.425290+0000 (oldest deadline 2023-10-07T23:18:39.325566+0000) 2023-10-07T23:19:03.256 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:02 smithi067 conmon[130383]: 2023-10-07T23:19:02.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:19.326498+0000 front 2023-10-07T23:18:19.326444+0000 (oldest deadline 2023-10-07T23:18:44.626260+0000) 2023-10-07T23:19:03.256 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:02 smithi067 conmon[130383]: 2023-10-07T23:19:02.844+0000 7f2145848700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.327756+0000 front 2023-10-07T23:18:29.327778+0000 (oldest deadline 2023-10-07T23:18:53.427602+0000) 2023-10-07T23:19:03.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:03 smithi123 conmon[160185]: 2023-10-07T23:19:03.513+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:03.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:03 smithi123 conmon[160185]: 2023-10-07T23:19:03.513+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:03.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:03 smithi123 conmon[160185]: 2023-10-07T23:19:03.513+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:03.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:03 smithi123 conmon[160185]: 2023-10-07T23:19:03.513+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:03.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:03 smithi123 conmon[160185]: 2023-10-07T23:19:03.513+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:03.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:03 smithi123 conmon[160185]: 2023-10-07T23:19:03.513+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:04.004 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:03 smithi067 bash[165897]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-9 2023-10-07T23:19:04.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:04.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:04.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:03 smithi086 conmon[158442]: 2023-10-07T23:19:03.879+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:04.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:04 smithi123 conmon[160185]: 2023-10-07T23:19:04.507+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:04.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:04 smithi123 conmon[160185]: 2023-10-07T23:19:04.507+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:04.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:04 smithi123 conmon[160185]: 2023-10-07T23:19:04.507+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:04.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:04 smithi123 conmon[160185]: 2023-10-07T23:19:04.507+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:04.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:04 smithi123 conmon[160185]: 2023-10-07T23:19:04.507+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:04.892 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:04 smithi123 conmon[160185]: 2023-10-07T23:19:04.507+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:05.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:05.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:05.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:05.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:05.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:05.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:05.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:04 smithi086 conmon[158442]: 2023-10-07T23:19:04.880+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:05.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-07T23:19:05.575 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:05 smithi067 systemd[1]: ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.9.service: Succeeded. 2023-10-07T23:19:05.575 INFO:journalctl@ceph.osd.9.smithi067.stdout:Oct 07 23:19:05 smithi067 systemd[1]: Stopped Ceph osd.9 for 227de690-6566-11ee-8db5-212e2dc638e7. 2023-10-07T23:19:05.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:05.601 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-07T23:19:05.601 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-07T23:19:05.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.10 2023-10-07T23:19:05.878 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 systemd[1]: Stopping Ceph osd.10 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:05.879 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:05.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:05 smithi123 conmon[160185]: 2023-10-07T23:19:05.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:05.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:05 smithi123 conmon[160185]: 2023-10-07T23:19:05.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:05.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:05 smithi123 conmon[160185]: 2023-10-07T23:19:05.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:05.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:05 smithi123 conmon[160185]: 2023-10-07T23:19:05.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:05.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:05 smithi123 conmon[160185]: 2023-10-07T23:19:05.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:05.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:05 smithi123 conmon[160185]: 2023-10-07T23:19:05.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:06.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:06.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:06.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:06.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:06.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:06.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.878+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:06.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.993+0000 7f82fea4f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:06.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.993+0000 7f82fea4f700 -1 osd.10 97 *** Got signal Terminated *** 2023-10-07T23:19:06.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:05 smithi086 conmon[158442]: 2023-10-07T23:19:05.993+0000 7f82fea4f700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:19:06.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:06 smithi123 conmon[160185]: 2023-10-07T23:19:06.544+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:06.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:06 smithi123 conmon[160185]: 2023-10-07T23:19:06.544+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:06.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:06 smithi123 conmon[160185]: 2023-10-07T23:19:06.544+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:06.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:06 smithi123 conmon[160185]: 2023-10-07T23:19:06.544+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:06.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:06 smithi123 conmon[160185]: 2023-10-07T23:19:06.544+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:06.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:06 smithi123 conmon[160185]: 2023-10-07T23:19:06.544+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:07.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:07.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:07.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:07.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:07.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:07.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:07.199 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:06 smithi086 conmon[158442]: 2023-10-07T23:19:06.861+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:07.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:07 smithi123 conmon[160185]: 2023-10-07T23:19:07.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:07.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:07 smithi123 conmon[160185]: 2023-10-07T23:19:07.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:07.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:07 smithi123 conmon[160185]: 2023-10-07T23:19:07.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:07.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:07 smithi123 conmon[160185]: 2023-10-07T23:19:07.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:07.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:07 smithi123 conmon[160185]: 2023-10-07T23:19:07.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:07.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:07 smithi123 conmon[160185]: 2023-10-07T23:19:07.588+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:08.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:08.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:08.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:08.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:08.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:08.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:08.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:07 smithi086 conmon[158442]: 2023-10-07T23:19:07.907+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:08.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:08.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:08.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:08.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:08.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:08.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:08.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:08 smithi123 conmon[160185]: 2023-10-07T23:19:08.583+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:09.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:09.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:09.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:09.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:09.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:09.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:09.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:08 smithi086 conmon[158442]: 2023-10-07T23:19:08.886+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:09.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:09.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:09.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:09.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:09.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:09.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:09.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:09 smithi123 conmon[160185]: 2023-10-07T23:19:09.548+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:10.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:10.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:10.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:10.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:10.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:10.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:10.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:09 smithi086 conmon[158442]: 2023-10-07T23:19:09.845+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:10.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:10.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:10.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:10.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:10.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:10.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:10.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:10 smithi123 conmon[160185]: 2023-10-07T23:19:10.541+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:11.197 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.138233+0000 front 2023-10-07T23:17:48.138279+0000 (oldest deadline 2023-10-07T23:18:13.438255+0000) 2023-10-07T23:19:11.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:57.539347+0000 front 2023-10-07T23:17:57.539277+0000 (oldest deadline 2023-10-07T23:18:18.639542+0000) 2023-10-07T23:19:11.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:03.940455+0000 front 2023-10-07T23:18:03.940386+0000 (oldest deadline 2023-10-07T23:18:26.840405+0000) 2023-10-07T23:19:11.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2023-10-07T23:18:12.141684+0000 front 2023-10-07T23:18:12.141571+0000 (oldest deadline 2023-10-07T23:18:37.441301+0000) 2023-10-07T23:19:11.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:21.542170+0000 front 2023-10-07T23:18:21.542224+0000 (oldest deadline 2023-10-07T23:18:45.042452+0000) 2023-10-07T23:19:11.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:27.343446+0000 front 2023-10-07T23:18:27.343494+0000 (oldest deadline 2023-10-07T23:18:51.443451+0000) 2023-10-07T23:19:11.198 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:10 smithi086 conmon[158442]: 2023-10-07T23:19:10.889+0000 7f82f8047700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:36.744691+0000 front 2023-10-07T23:18:36.744772+0000 (oldest deadline 2023-10-07T23:19:02.044597+0000) 2023-10-07T23:19:11.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:11.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:11.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:11.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:11.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:11.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:11.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:11 smithi123 conmon[160185]: 2023-10-07T23:19:11.586+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:12.447 INFO:journalctl@ceph.osd.10.smithi086.stdout:Oct 07 23:19:11 smithi086 bash[171867]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-10 2023-10-07T23:19:12.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:12.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:12.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:12.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:12.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:12.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:12.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:12 smithi123 conmon[160185]: 2023-10-07T23:19:12.559+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:13.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-07T23:19:13.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:13.693 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-07T23:19:13.693 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-07T23:19:13.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.11 2023-10-07T23:19:13.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 conmon[160185]: 2023-10-07T23:19:13.604+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:13.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:13 smithi123 systemd[1]: Stopping Ceph osd.11 for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:14.390 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.038+0000 7fd356e5f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:14.391 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.038+0000 7fd356e5f700 -1 osd.11 97 *** Got signal Terminated *** 2023-10-07T23:19:14.391 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.038+0000 7fd356e5f700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-07T23:19:14.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:14.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:14.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:14.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:14.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:14.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:14.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:14 smithi123 conmon[160185]: 2023-10-07T23:19:14.623+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:15.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:15.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:15.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:15.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:15.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:15.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:15.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:15 smithi123 conmon[160185]: 2023-10-07T23:19:15.584+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:16.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:16.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:16.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:16.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:16.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:16.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:16.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:16 smithi123 conmon[160185]: 2023-10-07T23:19:16.563+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:17.890 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:17.891 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:17 smithi123 conmon[160185]: 2023-10-07T23:19:17.605+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2023-10-07T23:18:53.088527+0000 front 2023-10-07T23:18:53.088495+0000 (oldest deadline 2023-10-07T23:19:17.188559+0000) 2023-10-07T23:19:19.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2023-10-07T23:17:48.775254+0000 front 2023-10-07T23:17:48.775286+0000 (oldest deadline 2023-10-07T23:18:11.675521+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2023-10-07T23:17:56.376962+0000 front 2023-10-07T23:17:56.376947+0000 (oldest deadline 2023-10-07T23:18:23.377415+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-10-07T23:18:05.678601+0000 front 2023-10-07T23:18:05.678796+0000 (oldest deadline 2023-10-07T23:18:28.578769+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2023-10-07T23:18:20.682198+0000 front 2023-10-07T23:18:20.682152+0000 (oldest deadline 2023-10-07T23:18:44.182254+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-10-07T23:18:29.884414+0000 front 2023-10-07T23:18:29.884345+0000 (oldest deadline 2023-10-07T23:18:55.784534+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2023-10-07T23:18:35.785303+0000 front 2023-10-07T23:18:35.785212+0000 (oldest deadline 2023-10-07T23:19:00.485150+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2023-10-07T23:18:45.686483+0000 front 2023-10-07T23:18:45.686524+0000 (oldest deadline 2023-10-07T23:19:07.887134+0000) 2023-10-07T23:19:19.334 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:18 smithi123 conmon[160185]: 2023-10-07T23:19:18.645+0000 7fd35245b700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2023-10-07T23:18:53.088527+0000 front 2023-10-07T23:18:53.088495+0000 (oldest deadline 2023-10-07T23:19:17.188559+0000) 2023-10-07T23:19:20.140 INFO:journalctl@ceph.osd.11.smithi123.stdout:Oct 07 23:19:19 smithi123 bash[173781]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-osd-11 2023-10-07T23:19:21.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-07T23:19:21.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:21.586 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-07T23:19:21.586 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-07T23:19:21.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.a 2023-10-07T23:19:22.004 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:19:21 smithi067 systemd[1]: Stopping Ceph mds.a for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:22.504 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:19:22 smithi067 conmon[139842]: 2023-10-07T23:19:22.097+0000 7fc55f40e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:22.505 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:19:22 smithi067 conmon[139842]: 2023-10-07T23:19:22.097+0000 7fc55f40e700 -1 mds.a *** got signal Terminated *** 2023-10-07T23:19:24.937 INFO:journalctl@ceph.mds.a.smithi067.stdout:Oct 07 23:19:24 smithi067 bash[166445]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-a 2023-10-07T23:19:24.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-07T23:19:25.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:25.041 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-07T23:19:25.041 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-07T23:19:25.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.d 2023-10-07T23:19:25.254 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:19:25 smithi067 systemd[1]: Stopping Ceph mds.d for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:26.004 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:19:25 smithi067 conmon[138389]: 2023-10-07T23:19:25.537+0000 7f9b3a02e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:26.005 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:19:25 smithi067 conmon[138389]: 2023-10-07T23:19:25.537+0000 7f9b3a02e700 -1 mds.d *** got signal Terminated *** 2023-10-07T23:19:28.927 INFO:journalctl@ceph.mds.d.smithi067.stdout:Oct 07 23:19:28 smithi067 bash[166727]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-d 2023-10-07T23:19:28.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-07T23:19:29.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:29.031 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-07T23:19:29.031 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-07T23:19:29.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.g 2023-10-07T23:19:29.254 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:19:29 smithi067 systemd[1]: Stopping Ceph mds.g for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:30.004 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:19:29 smithi067 conmon[139201]: 2023-10-07T23:19:29.520+0000 7fc4d2943700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:30.005 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:19:29 smithi067 conmon[139201]: 2023-10-07T23:19:29.520+0000 7fc4d2943700 -1 mds.g *** got signal Terminated *** 2023-10-07T23:19:31.254 INFO:journalctl@ceph.mds.g.smithi067.stdout:Oct 07 23:19:30 smithi067 bash[167010]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-g 2023-10-07T23:19:31.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-07T23:19:31.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:31.384 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-07T23:19:31.385 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-07T23:19:31.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.j 2023-10-07T23:19:31.754 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:19:31 smithi067 systemd[1]: Stopping Ceph mds.j for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:32.254 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:19:31 smithi067 conmon[137786]: 2023-10-07T23:19:31.879+0000 7ffa546ae700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:32.254 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:19:31 smithi067 conmon[137786]: 2023-10-07T23:19:31.879+0000 7ffa546ae700 -1 mds.j *** got signal Terminated *** 2023-10-07T23:19:35.900 INFO:journalctl@ceph.mds.j.smithi067.stdout:Oct 07 23:19:35 smithi067 bash[167292]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-j 2023-10-07T23:19:35.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-07T23:19:36.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:36.046 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-07T23:19:36.046 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-07T23:19:36.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.b 2023-10-07T23:19:36.379 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:19:36 smithi086 systemd[1]: Stopping Ceph mds.b for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:36.697 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:19:36 smithi086 conmon[166399]: 2023-10-07T23:19:36.377+0000 7fcded19f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:36.697 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:19:36 smithi086 conmon[166399]: 2023-10-07T23:19:36.377+0000 7fcded19f700 -1 mds.b *** got signal Terminated *** 2023-10-07T23:19:40.928 INFO:journalctl@ceph.mds.b.smithi086.stdout:Oct 07 23:19:40 smithi086 bash[172418]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-b 2023-10-07T23:19:40.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-07T23:19:40.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:40.986 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-07T23:19:40.986 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-07T23:19:40.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.e 2023-10-07T23:19:41.324 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:19:41 smithi086 systemd[1]: Stopping Ceph mds.e for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:41.697 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:19:41 smithi086 conmon[165267]: 2023-10-07T23:19:41.322+0000 7fab8fa1c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:41.697 INFO:journalctl@ceph.mds.e.smithi086.stdout:Oct 07 23:19:41 smithi086 conmon[165267]: 2023-10-07T23:19:41.322+0000 7fab8fa1c700 -1 mds.e *** got signal Terminated *** 2023-10-07T23:19:45.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-07T23:19:45.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:45.093 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-07T23:19:45.093 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-07T23:19:45.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.h 2023-10-07T23:19:45.447 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:19:45 smithi086 systemd[1]: Stopping Ceph mds.h for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:45.947 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:19:45 smithi086 conmon[165830]: 2023-10-07T23:19:45.450+0000 7f5577c9c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:45.947 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:19:45 smithi086 conmon[165830]: 2023-10-07T23:19:45.450+0000 7f5577c9c700 -1 mds.h *** got signal Terminated *** 2023-10-07T23:19:48.697 INFO:journalctl@ceph.mds.h.smithi086.stdout:Oct 07 23:19:48 smithi086 bash[172977]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-h 2023-10-07T23:19:48.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-07T23:19:49.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:49.020 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-07T23:19:49.020 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-07T23:19:49.020 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.k 2023-10-07T23:19:49.356 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:19:49 smithi086 systemd[1]: Stopping Ceph mds.k for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:49.697 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:19:49 smithi086 conmon[164704]: 2023-10-07T23:19:49.356+0000 7f6c4c090700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:49.698 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:19:49 smithi086 conmon[164704]: 2023-10-07T23:19:49.356+0000 7f6c4c090700 -1 mds.k *** got signal Terminated *** 2023-10-07T23:19:52.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:Oct 07 23:19:52 smithi086 bash[173265]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-k 2023-10-07T23:19:53.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-07T23:19:53.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:53.036 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-07T23:19:53.036 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-07T23:19:53.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.c 2023-10-07T23:19:53.390 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:19:53 smithi123 systemd[1]: Stopping Ceph mds.c for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:53.890 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:19:53 smithi123 conmon[167744]: 2023-10-07T23:19:53.399+0000 7ff0b5ce8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:53.891 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:19:53 smithi123 conmon[167744]: 2023-10-07T23:19:53.399+0000 7ff0b5ce8700 -1 mds.c *** got signal Terminated *** 2023-10-07T23:19:56.890 INFO:journalctl@ceph.mds.c.smithi123.stdout:Oct 07 23:19:56 smithi123 bash[174334]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-c 2023-10-07T23:19:57.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-07T23:19:57.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:19:57.133 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-07T23:19:57.133 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-07T23:19:57.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.f 2023-10-07T23:19:57.390 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:19:57 smithi123 systemd[1]: Stopping Ceph mds.f for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:19:57.890 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:19:57 smithi123 conmon[167184]: 2023-10-07T23:19:57.486+0000 7f2fb0745700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:19:57.890 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:19:57 smithi123 conmon[167184]: 2023-10-07T23:19:57.486+0000 7f2fb0745700 -1 mds.f *** got signal Terminated *** 2023-10-07T23:20:00.623 INFO:journalctl@ceph.mds.f.smithi123.stdout:Oct 07 23:20:00 smithi123 bash[174618]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-f 2023-10-07T23:20:01.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-07T23:20:01.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:20:01.128 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-07T23:20:01.128 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-07T23:20:01.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.i 2023-10-07T23:20:01.390 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:20:01 smithi123 systemd[1]: Stopping Ceph mds.i for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:20:01.890 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:20:01 smithi123 conmon[168307]: 2023-10-07T23:20:01.485+0000 7f5b6cbdf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:20:01.891 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:20:01 smithi123 conmon[168307]: 2023-10-07T23:20:01.485+0000 7f5b6cbdf700 -1 mds.i *** got signal Terminated *** 2023-10-07T23:20:04.890 INFO:journalctl@ceph.mds.i.smithi123.stdout:Oct 07 23:20:04 smithi123 bash[174904]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-i 2023-10-07T23:20:04.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-07T23:20:05.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:20:05.025 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-07T23:20:05.025 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-07T23:20:05.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.l 2023-10-07T23:20:05.387 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:20:05 smithi123 systemd[1]: Stopping Ceph mds.l for 227de690-6566-11ee-8db5-212e2dc638e7... 2023-10-07T23:20:05.640 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:20:05 smithi123 conmon[166622]: 2023-10-07T23:20:05.386+0000 7f6383a2f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-07T23:20:05.641 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:20:05 smithi123 conmon[166622]: 2023-10-07T23:20:05.386+0000 7f6383a2f700 -1 mds.l *** got signal Terminated *** 2023-10-07T23:20:08.890 INFO:journalctl@ceph.mds.l.smithi123.stdout:Oct 07 23:20:08 smithi123 bash[175186]: ceph-227de690-6566-11ee-8db5-212e2dc638e7-mds-l 2023-10-07T23:20:09.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-227de690-6566-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-07T23:20:09.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-07T23:20:09.137 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-07T23:20:09.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-07T23:20:13.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-07T23:20:14.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-07T23:20:16.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:20:16.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:20:17.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-07T23:20:17.053 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-07T23:20:17.055 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi067/crash 2023-10-07T23:20:17.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash -- . 2023-10-07T23:20:17.130 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-07T23:20:17.131 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2023-10-07T23:20:17.134 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi086/crash 2023-10-07T23:20:17.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash -- . 2023-10-07T23:20:17.198 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-07T23:20:17.198 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2023-10-07T23:20:17.200 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi123/crash 2023-10-07T23:20:17.201 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash -- . 2023-10-07T23:20:17.238 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/227de690-6566-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-07T23:20:17.238 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2023-10-07T23:20:17.239 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-07T23:20:17.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-07T23:20:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:grep: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-07T23:20:17.317 INFO:tasks.cephadm:Compressing logs... 2023-10-07T23:20:17.317 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T23:20:17.322 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T23:20:17.324 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T23:20:17.347 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-07T23:20:17.347 INFO:teuthology.orchestra.run.smithi123.stderr:: No such file or directory 2023-10-07T23:20:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-07T23:20:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-07T23:20:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-07T23:20:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-07T23:20:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-07T23:20:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-07T23:20:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-07T23:20:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:: No such file or directory 2023-10-07T23:20:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-07T23:20:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-07T23:20:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-07T23:20:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-07T23:20:17.351 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-07T23:20:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.10.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-07T23:20:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-07T23:20:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-07T23:20:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.4.log: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.1.log: 90.9% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-07T23:20:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-07T23:20:17.355 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-07T23:20:17.359 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-07T23:20:17.359 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.f.log: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.c.log: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.11.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-07T23:20:17.359 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-07T23:20:17.359 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.c.log: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-07T23:20:17.359 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-07T23:20:17.361 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: 91.7% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-07T23:20:17.361 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-07T23:20:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-07T23:20:17.364 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.5.log: 92.9% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-07T23:20:17.364 INFO:teuthology.orchestra.run.smithi123.stderr: 91.6% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-07T23:20:17.365 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-07T23:20:17.365 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-07T23:20:17.365 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-07T23:20:17.366 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: 91.7% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-07T23:20:17.366 INFO:teuthology.orchestra.run.smithi086.stderr: 92.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-07T23:20:17.369 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.2.log: 91.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-07T23:20:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.e.log: 91.3% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-07T23:20:17.377 INFO:teuthology.orchestra.run.smithi086.stderr: 92.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-07T23:20:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-07T23:20:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:: No such file or directory 2023-10-07T23:20:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-07T23:20:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-07T23:20:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-07T23:20:17.383 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-07T23:20:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-07T23:20:17.386 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-07T23:20:17.386 INFO:teuthology.orchestra.run.smithi067.stderr: 94.7% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-07T23:20:17.387 INFO:teuthology.orchestra.run.smithi067.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-07T23:20:17.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-07T23:20:17.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.9.log: 90.9% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-07T23:20:17.389 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-07T23:20:17.391 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-07T23:20:17.393 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-07T23:20:17.394 INFO:teuthology.orchestra.run.smithi086.stderr: 93.2% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-07T23:20:17.395 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-07T23:20:17.395 INFO:teuthology.orchestra.run.smithi067.stderr: 92.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-07T23:20:17.397 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-07T23:20:17.401 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-07T23:20:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.log: 91.5% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-07T23:20:17.406 INFO:teuthology.orchestra.run.smithi067.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-07T23:20:17.531 INFO:teuthology.orchestra.run.smithi086.stderr: 91.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-07T23:20:17.531 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-07T23:20:17.546 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-07T23:20:17.598 INFO:teuthology.orchestra.run.smithi123.stderr: 92.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-07T23:20:17.601 INFO:teuthology.orchestra.run.smithi123.stderr: 92.3% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-07T23:20:17.618 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-07T23:20:17.639 INFO:teuthology.orchestra.run.smithi067.stderr: 92.2% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-07T23:20:17.655 INFO:teuthology.orchestra.run.smithi123.stderr: 92.0% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-07T23:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-07T23:20:17.670 INFO:teuthology.orchestra.run.smithi086.stderr: 92.2% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-07T23:20:17.674 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-07T23:20:17.680 INFO:teuthology.orchestra.run.smithi086.stderr: 92.0% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-07T23:20:17.682 INFO:teuthology.orchestra.run.smithi086.stderr: 2023-10-07T23:20:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m0.345s 2023-10-07T23:20:17.682 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m1.458s 2023-10-07T23:20:17.683 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.059s 2023-10-07T23:20:17.704 INFO:teuthology.orchestra.run.smithi067.stderr: 92.2% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-07T23:20:17.714 INFO:teuthology.orchestra.run.smithi123.stderr: 92.2% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-07T23:20:17.716 INFO:teuthology.orchestra.run.smithi123.stderr: 2023-10-07T23:20:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m0.379s 2023-10-07T23:20:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m1.367s 2023-10-07T23:20:17.716 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.062s 2023-10-07T23:20:17.734 INFO:teuthology.orchestra.run.smithi067.stderr: 92.4% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-07T23:20:17.941 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/227de690-6566-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-07T23:20:17.944 INFO:teuthology.orchestra.run.smithi067.stderr: 2023-10-07T23:20:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m0.593s 2023-10-07T23:20:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m1.814s 2023-10-07T23:20:17.945 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m0.216s 2023-10-07T23:20:17.946 INFO:tasks.cephadm:Archiving logs... 2023-10-07T23:20:17.946 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi067/log 2023-10-07T23:20:17.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T23:20:18.254 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi086/log 2023-10-07T23:20:18.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T23:20:18.480 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi123/log 2023-10-07T23:20:18.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T23:20:18.695 INFO:tasks.cephadm:Removing cluster... 2023-10-07T23:20:18.696 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --force 2023-10-07T23:20:19.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --force 2023-10-07T23:20:20.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 227de690-6566-11ee-8db5-212e2dc638e7 --force 2023-10-07T23:20:21.433 INFO:tasks.cephadm:Removing cephadm ... 2023-10-07T23:20:21.433 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-07T23:20:21.475 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-07T23:20:21.493 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-07T23:20:21.507 INFO:tasks.cephadm:Teardown complete 2023-10-07T23:20:21.507 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-07T23:20:21.516 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-07T23:20:21.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T23:20:21.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T23:20:21.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T23:20:21.653 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-07T23:20:21.663 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-07T23:20:21.663 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:20:21.669 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:20:21.671 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:20:21.684 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-10-07T23:20:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2023-10-07T23:20:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:210 Number of sources = 4 2023-10-07T23:20:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T23:20:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-10-07T23:20:21.689 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 23 -367us[ -367us] +/- 82ms 2023-10-07T23:20:21.689 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 +147us[ +159us] +/- 62ms 2023-10-07T23:20:21.689 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +153us[ +165us] +/- 67ms 2023-10-07T23:20:21.689 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:20:21.690 INFO:teuthology.orchestra.run.smithi086.stdout:210 Number of sources = 4 2023-10-07T23:20:21.690 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T23:20:21.690 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2023-10-07T23:20:21.690 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 25 -360us[ -353us] +/- 82ms 2023-10-07T23:20:21.691 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 +157us[ +165us] +/- 61ms 2023-10-07T23:20:21.691 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +153us[ +160us] +/- 67ms 2023-10-07T23:20:21.691 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:20:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2023-10-07T23:20:21.705 INFO:teuthology.orchestra.run.smithi067.stdout:210 Number of sources = 4 2023-10-07T23:20:21.705 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-07T23:20:21.706 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-10-07T23:20:21.706 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -387us[ -387us] +/- 82ms 2023-10-07T23:20:21.706 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 +156us[ +156us] +/- 61ms 2023-10-07T23:20:21.706 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +140us[ +140us] +/- 67ms 2023-10-07T23:20:21.706 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-07T23:20:21.707 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-07T23:20:21.730 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-07T23:20:21.731 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-07T23:20:21.792 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-07T23:20:21.965 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-07T23:20:22.061 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-07T23:20:22.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T23:20:22.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:22.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T23:20:22.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:22.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-07T23:20:22.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:22.279 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-07T23:20:22.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-07T23:20:22.299 INFO:teuthology.task.internal:Duration was 1695.952006 seconds 2023-10-07T23:20:22.299 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-07T23:20:22.312 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-07T23:20:22.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T23:20:22.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T23:20:22.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T23:20:22.361 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T23:20:22.362 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T23:20:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-07T23:20:22.708 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-07T23:20:22.709 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2023-10-07T23:20:22.709 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T23:20:22.761 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2023-10-07T23:20:22.761 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T23:20:22.781 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2023-10-07T23:20:22.781 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T23:20:22.801 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-07T23:20:22.801 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T23:20:22.806 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T23:20:22.823 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T23:20:22.893 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-07T23:20:22.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T23:20:22.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T23:20:22.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T23:20:23.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-07T23:20:23.056 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-07T23:20:23.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T23:20:23.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T23:20:23.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T23:20:23.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-07T23:20:23.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:20:23.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:20:23.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:20:23.204 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2023-10-07T23:20:23.227 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2023-10-07T23:20:23.236 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2023-10-07T23:20:23.272 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:20:23.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:23.307 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:20:23.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:23.320 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:20:23.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:23.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-07T23:20:23.344 INFO:teuthology.task.internal:Transferring archived files... 2023-10-07T23:20:23.344 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi067 2023-10-07T23:20:23.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T23:20:23.424 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi086 2023-10-07T23:20:23.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T23:20:23.464 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034/remote/smithi123 2023-10-07T23:20:23.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T23:20:23.500 INFO:teuthology.task.internal:Removing archive directory... 2023-10-07T23:20:23.501 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T23:20:23.506 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T23:20:23.508 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T23:20:23.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-07T23:20:23.559 INFO:teuthology.task.internal:Not uploading archives. 2023-10-07T23:20:23.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-07T23:20:23.581 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-07T23:20:23.581 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T23:20:23.586 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T23:20:23.588 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T23:20:23.600 INFO:teuthology.orchestra.run.smithi086.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 23:20 /home/ubuntu/cephtest 2023-10-07T23:20:23.601 INFO:teuthology.orchestra.run.smithi123.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 23:20 /home/ubuntu/cephtest 2023-10-07T23:20:23.618 INFO:teuthology.orchestra.run.smithi067.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 23:20 /home/ubuntu/cephtest 2023-10-07T23:20:23.621 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-07T23:20:23.632 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-07T23:20:23.653 INFO:teuthology.nuke:Checking targets against current locks 2023-10-07T23:20:23.677 DEBUG:teuthology.nuke:shortname: smithi067 2023-10-07T23:20:23.677 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:20:23.699 DEBUG:teuthology.nuke:shortname: smithi086 2023-10-07T23:20:23.699 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:20:23.704 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 22:43:02.410449', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3iQMWZ//AVUYsGaoIawOqjB+7cK8RVEAe5L5kvT9lAPyUCOkuh8SHyaWo+9X83DY15w2jV4sIrN2MgSyzCL5E='} 2023-10-07T23:20:23.722 DEBUG:teuthology.nuke:shortname: smithi123 2023-10-07T23:20:23.722 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:20:23.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 22:43:02.411961', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFRLp2JeOCbBLAfSQ/QSN91MIiIotZlxeJx7RvTI6mYzKBrV3Ngh9zhXujL2kieOSDrPoYTZyBx9o3gWn2IJM4M='} 2023-10-07T23:20:23.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 22:43:02.408701', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPxdMWiYGXZ9jBVTUMzUA/sdy5F4FxcZoC2B/rds4X0S5vRwRjolYZD6OUy6XBHJfRxzaJwl0rSqzpl80/k/r7Q='} 2023-10-07T23:20:23.756 INFO:teuthology.orchestra.console.smithi067:Power off 2023-10-07T23:20:23.756 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T23:20:23.778 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2023-10-07T23:20:23.778 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:23.791 INFO:teuthology.orchestra.console.smithi086:Power off 2023-10-07T23:20:23.792 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T23:20:23.805 INFO:teuthology.orchestra.console.smithi123:Power off 2023-10-07T23:20:23.806 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T23:20:23.814 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-10-07T23:20:23.814 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2023-10-07T23:20:23.814 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:23.827 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-10-07T23:20:23.827 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:23.836 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-07T23:20:23.848 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-10-07T23:20:27.814 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:27.838 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:27.848 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:27.924 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-10-07T23:20:27.947 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-07T23:20:27.957 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-10-07T23:20:31.926 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:31.948 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:31.958 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:20:32.037 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2023-10-07T23:20:32.058 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-10-07T23:20:32.067 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-10-07T23:20:32.138 INFO:teuthology.orchestra.console.smithi067:Power off completed 2023-10-07T23:20:32.158 INFO:teuthology.orchestra.console.smithi086:Power off completed 2023-10-07T23:20:32.168 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-10-07T23:20:32.273 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} duration: 1695.9520061016083 failure_reason: '' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1e7a0e5812964784baf5224c7ca88154 status: fail success: false 2023-10-07T23:20:32.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T23:20:32.362 INFO:teuthology.run:FAIL